Monday, June 5, 2023

The very strange case of the AWS Workspace Time Zone slip to UTC

 


From implementation the AWS Workspaces exhibited a problem with random time zone slip to the Universal Time Co-ordinated (UTC) time zone.  For UK users this was only really noticeable during the summer when Greenwich Mean Time (GMT) Daylight saving was applied.  In this case the correct time in summer is British Summer Time (BST) = GMT +1.  Random slips back to UTC caused the system time to be 1 hour behind the required BST.  In the UK winter GMT = UTC so no one in the UK noticed the effect during the winter months.  One of the main impacts was the potential for meetings to be missed because the computer would fail to produce reminders for imminent meetings. 

Users in other time zones would also encounter the time zone slip to UTC.

The slip to UTC was closely monitored and appeared to be completely random.  Initially the Windows Time Service client configurations were updated which reduced the time zone slip events but did not completely remove all of them.

The root cause of the time zone slip could not be determined and analysis was hindered by the random nature of the problem and the general complexity of Windows Time configuration.  The suspicion was that the AWS Workspace with two network adapters was somehow obtaining time information from an Amazon network device using UTC. An active correction solution was designed to detect the slip and correct it.

The time zone on an AWS Workspace is configured during the AWS Client connection process.  The AWS Client uses the time zone of the computer it is running on and applies it to the AWS Workspace.  After connecting to an AWS Workspace a user always has the correct time local to them, until a UTC slip event occurs.

The solution uses powershell scripts and event triggered scheduled tasks delivered by GPO.  These log the correct time zone during connection and then reapply it if a UTC time zone slip event is detected.  The correction occurs instantaneously and is unnoticeable by the logged in user.  

When the AWS Workspaces Client is used to connect to a AWS Workspace computer an event 88 is logged in the Application Event Log.  This process also sets the workspace time zone to match the time zone of the system the AWS Client software is running on.  Event 88 triggers a call to the Log-Timezone.ps1 script which writes the currently correct time zone to a text file C:\Windows\Logs\TZStore.log

$TZStore=$env:SystemRoot+"\Logs\TZStore.log"

#$TZStore Start-Sleep -S 1 $TZ=invoke-command -command {c:\windows\system32\tzutil /g} | out-file $TZStore

Time zone changing events are logged as Microsoft-Windows-Kernel-General event 1 in the System Event Log.  This triggers a call to Check-Timezone.ps1.  If the incorrect UTC time zone is detected, the script obtains the correct time zone from the TZStore.log file and applies it.

<#

This script has just been triggered by a Kernel-General EventID 1 (Change Reason: System time adjusted to the new time zone.) Check Timezone matches the Timezone setting logged at AWS Client connection. #> $ErrorActionPreference= 'silentlycontinue' #Find required Timezone $LoggedTZ=$null $TZStore=$env:SystemRoot+"\Logs\TZStore.log" $LoggedTZ = Get-Content $TZStore #exit if the TZ store is not there. if ($LoggedTZ -eq $null){Exit} #Set log for corrections $TZCorrections=$env:SystemRoot+"\Logs\TZCorrections.log" $date=Get-Date #Check and correct TZ if required. $TZ=invoke-command -command {c:\windows\system32\tzutil /g} if ($TZ -like "UTC") { invoke-command -command {c:\windows\system32\tzutil /s “$LoggedTZ”} echo "Changed UTC to $LoggedTZ at $date" | out-file -append $TZCorrections }

Refer to this post for more information about configuring Scheduled Tasks triggered by event log entries

https://davespshell.blogspot.com/2021/03/automatically-logoff-idle-users.html