<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Source Sans Pro";
panose-1:2 11 5 3 3 4 3 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
mso-add-space:auto;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;}
p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
mso-add-space:auto;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;}
p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
mso-add-space:auto;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;}
p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
mso-add-space:auto;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:232938114;
mso-list-template-ids:-49276988;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1
{mso-list-id:319115535;
mso-list-type:hybrid;
mso-list-template-ids:-18609004 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l2
{mso-list-id:1196701123;
mso-list-template-ids:1943582100;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3
{mso-list-id:1956907272;
mso-list-type:hybrid;
mso-list-template-ids:1974494284 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l3:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l3:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l3:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l3:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l3:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l3:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l3:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l3:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l3:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Dear COE community,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I share the note (below) from VP and Chief Information Officer, Abhijit Pandit, about this week’s technology outage that affected many of us. For those of us who were affected, it was pretty debilitating (and
happened at a rough time of the academic year!). Please see below for a description of the problem and key action items.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Best,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Laura Lee <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:.25in;margin-right:0in;margin-bottom:.25in;margin-left:0in;background:white">
<b><span style="font-family:"Source Sans Pro",sans-serif;color:black;mso-ligatures:none">Laura Lee McIntyre | Dean & Castle-McIntosh-Knight Professor</span></b><span style="font-family:"Source Sans Pro",sans-serif;color:black;mso-ligatures:none"> </span><span style="font-size:11.0pt;font-family:"Source Sans Pro",sans-serif;color:black;mso-ligatures:none">(she/her)</span><span style="font-family:"Source Sans Pro",sans-serif;color:black;mso-ligatures:none"><br>
</span><b><span style="font-size:11.0pt;font-family:"Source Sans Pro",sans-serif;color:#737373;mso-ligatures:none">University of Oregon | College of Education</span></b><span style="font-family:"Source Sans Pro",sans-serif;color:black;mso-ligatures:none"><br>
</span><span style="font-size:11.0pt;font-family:"Source Sans Pro",sans-serif;color:black;mso-ligatures:none">230 Hedco Education Building | Eugene, OR 97403-1215<br>
<a href="mailto:email@uoregon.edu"><span style="color:blue">llmcinty@uoregon.edu</span></a> |
<a href="https://education.uoregon.edu/"><span style="color:blue">https://education.uoregon.edu</span></a></span><span style="font-family:"Source Sans Pro",sans-serif;color:black;mso-ligatures:none">
<o:p></o:p></span></p>
<p class="MsoNormal"><a href="https://www.uoregon.edu/"><span style="font-size:13.5pt;font-family:"Arial",sans-serif;color:blue;background:white;mso-ligatures:none;text-decoration:none"><img border="0" width="150" height="21" style="width:1.5625in;height:.2187in" id="Picture_x0020_1" src="cid:image001.png@01D99A2A.DDC8F960"></span></a><span style="font-size:11.0pt;mso-ligatures:none"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;mso-ligatures:none">From:</span></b><span style="font-size:11.0pt;mso-ligatures:none"> aallist-bounces@lists.uoregon.edu <aallist-bounces@lists.uoregon.edu>
<b>On Behalf Of </b>Chief Information Officer<br>
<b>Sent:</b> Thursday, June 8, 2023 4:52 PM<br>
<b>To:</b> aallist@lists.uoregon.edu<br>
<b>Subject:</b> [AALList] Update on this week's IT outages<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Dear colleagues,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This week, the University of Oregon has experienced an unusual technology outage affecting thousands of people for multiple days. I understand how disruptive it has been to have some of our most fundamental services unavailable or degraded
during such a critical time in the academic year. Please accept my sincere apologies.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I want to provide information to help support you and your teams while Information Services wraps up our cleanup work.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>Key Action Items</b><o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoListParagraphCxSpFirst" style="margin-left:0in;mso-add-space:auto;mso-list:l1 level1 lfo3">
Please <b>share this information</b> with colleagues who have been affected by this outage.
<o:p></o:p></li><li class="MsoListParagraphCxSpMiddle" style="margin-left:0in;mso-add-space:auto;mso-list:l1 level1 lfo3">
Visit the <a href="https://status.uoregon.edu/">IT Service Status</a> website for the latest
<a name="_Int_WygNYLxs">updates, and</a> encourage your teams to do the same — for this outage and any others.
<o:p></o:p></li><li class="MsoListParagraphCxSpLast" style="margin-left:0in;mso-add-space:auto;mso-list:l1 level1 lfo3">
Consider <b>forwarding</b> other <a name="_Int_KPLDEzVn">UO</a> announcements, such as Tuesday's
<a href="https://urldefense.com/v3/__https:/t.e2ma.net/webview/0lccuh/7a81b2f1cd0fe195b4f6d68ee20e3f76__;!!C5qS4YX3!DsvO1yBTTUhjSrnavUD7qLFDrR3WD3ppl7vyRnrc4qr6584T4kOEMopyWJer8NP9B2SpeN4PPO2N_-0D$">
Around the O Workplace newsletter</a>, to affected colleagues.<o:p></o:p></li></ul>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>What Happened<o:p></o:p></b></p>
<p class="MsoNormal">The problem began on Monday, June 5, around 8:00 p.m. Pacific Time, when our new Duck ID system vendor, Bravura, performed planned maintenance. That work had been requested by our staff to address performance issues. Despite testing by
both UO and the vendor, an unexpected bug ultimately revoked service access and broke inbound email flow for about 4,600 UO accounts.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Those affected had problems accessing or using UOmail, Teams, and other Microsoft services, and couldn't log in to the UO Service Portal to submit help tickets. About 1,600 people lost Teams calling service, and some people lost access
to Dropbox.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">(Note: This was unrelated to Monday's widespread Microsoft outages that intermittently affected services such as UOmail and Teams at the UO and across the United States. Dropbox is also now reporting an unrelated partial outage.)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>Remaining Impacts<o:p></o:p></b></p>
<p class="MsoNormal">Most of the impacts of this outage had been resolved by yesterday morning (Wednesday, June 7). Others have just been resolved today, and a few still remain.
<i>Please check the </i><a href="https://status.uoregon.edu/"><i>IT Service Status</i></a><i> website for the most recent updates.</i><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoListParagraphCxSpFirst" style="margin-left:0in;mso-add-space:auto;mso-list:l3 level1 lfo6">
<b>Incoming email flow. </b>For the vast majority of affected accounts, mail flow was fully restored early yesterday morning (Wednesday, June 7). For a small number of accounts that remained affected, inbound mail flow was restored early this morning (Thursday,
June 8). <o:p></o:p></li><li class="MsoListParagraphCxSpMiddle" style="margin-left:0in;mso-add-space:auto;mso-list:l3 level1 lfo6">
<b>Emails sent to affected accounts. </b>During the outage (mainly on Tuesday),<b>
</b>some emails sent to affected accounts bounced. However, other messages ended up in a limbo state within our systems. Most messages in the latter category were delivered into people's mailboxes in a process completed around 12:30 p.m. today. Our team is
now doing a round of extended processing to ensure all affected users receive such emails. Depending on your email program, the messages may appear with either today’s date or their original send dates.<o:p></o:p></li><li class="MsoListParagraphCxSpMiddle" style="margin-left:0in;mso-add-space:auto;mso-list:l3 level1 lfo6">
<a name="_Int_2dnBhoCu"><b>Teams</b></a><b> calling. </b>Service has been restored for those affected. If anyone with Teams calling still can’t see their dial pad, they should log out of Teams and log back in again.<o:p></o:p></li><li class="MsoListParagraphCxSpLast" style="margin-left:0in;mso-add-space:auto;line-height:105%;mso-list:l3 level1 lfo6">
<b>Teams private channel access.</b> People who've lost access to private Teams channels can have their access restored by the owners or members of those channels. If that's not possible, please submit a ticket at
<a href="https://service.uoregon.edu/TDClient/2030/Portal/Requests/ServiceDet?ID=32268">
Microsoft Teams support</a>.<o:p></o:p></li></ul>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>Preventing a Recurrence</b><o:p></o:p></p>
<p class="MsoNormal">To prevent a recurrence at this critical time of year, we have paused any further work by Bravura until we better understand what went wrong on Monday. Once we resume work on the Duck ID system, Information Services will use our existing
internal review and planning processes to limit disruptions to the university's academic and business functions.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I want to thank the project team that has worked hard for many months to
<a href="https://around.uoregon.edu/content/new-duck-id-platform-will-overhaul-account-management">
launch our new Duck ID system</a> in February and make dozens of post-launch improvements. It would be hard to overstate the complexity of switching identity management systems at a university. Our Duck ID system manages a constant flux in affiliations and
access for more than 70,000<span style="color:red"> </span>members of the UO community. After some early bumps in the road, the team has worked diligently with the vendor to refine and stabilize the system. The UO is benefiting from the greater security and
self-service options of this new system, and more improvements to the user experience are on the docket for this summer.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If you or your teams need help relating to this outage, please contact the
<a href="https://service.uoregon.edu/TDClient/2030/Portal/KB/ArticleDet?ID=101085">
IT staff who support your unit</a> or the <a href="https://service.uoregon.edu/TDClient/2030/Portal/KB/ArticleDet?ID=31704">
Technology Service Desk</a>.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="line-height:105%">Sincerely,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Abhijit Pandit<o:p></o:p></p>
<p class="MsoNormal">Vice President and Chief Information Officer<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>