All I thought I would share a script that a colleague and I wrote, that I have gotten to automate with SAM when a reboot alert is triggered.
Breakdown
1. A person, robot, aliens, etc .... reboot a server that we have monitored.
2. We get an alert stating that the server has been rebooted and we get specifics on the reboot.
3. If we know about the reboot, we acknowledge the alert within 2 minutes and that is the end
4. If we don't acknowledge the alert, I then have a powershell script that goes out and pulls the 1074 events from the event manager on that server and emails the output to our team. We then know who rebooted and if they are kind enough to say why, then the why. This makes it much easier.
Here is an actual email. Names changed of course to protect the innocent.
**********************************************
From:SysAdmin@domain.org
Sent: Dec 20, 2015 2:24 PM
To: DataCenter Team
Subject: Reboot Event Review - serverblah.domain.local
Server Event Log Restart Parsed Results
EventID : 1074
MachineName : serverblah.domain.local
Message : The process C:\Windows\system32\winlogon.exe (serverblah.domain.local) has
initiated the restart of computer serverblah.domain.local on behalf of
user Domain\user blah for the following reason: Software upgrade
reason could be found
Reason Code: 0x500ff
Shutdown Type: restart
**************************************
Now the script and how I got this to work (Nasty solarwinds bug didn't make this easy)
1. Setup the script - Put the following into a notepad and name it and end it with .ps1 like autoreboot.ps1. Set the execution policy to passive. Then attempt to run the script from powershell - ./autoreboot.ps1 serverbla
***Remember to change the perimeters in the script to match your settings ****
*******************************************
#Reboot Check by Santez K / Aaron B - All credit goes to Santez K for getting foundation of this this script laid.
#IMPORTANT !!! If this stops working while using SAM alerting - Check the Solarwinds Orion Module service - It should be using a valid AD admin account.
param(
[string]$computer
)
Write-Host $computer
$after = (get-date).addminutes(-4000)
$Log = Get-EventLog -LogName system -after $after -ComputerName $computer | Where-Object {$_.EventID -eq 1074} | format-list -Property EventID, MachineName, Message, TimeGenerated, TimeWritten, UserName | out-string
Send-MailMessage -To "email@blah.org" -Subject "Reboot Event Review - $computer" -Body "Server Event Log Restart Parsed Results $Log" -smtpserver email.blah.local -From "email@blah.org"
***********************************
2. This is the most important part. Solarwinds alerting has a very nasty bug that won't do Windows authentication when setting up the alert. No matter what you use, it will say ummm no. I will explain how to get around this in the next bullet.
This is how I have the command set to run the script. Node name puts the server name in for the server it is alerting on.
3. For authentication you need to go to services on the SAM server and change the "Solarwinds Orion Module" to run as and pick the user who has keys to the world. We have a solarwinds service account set in AD for this so that is what we use. Weirdly that is what is used to run the script against. If you left it with local user, it would never authenticate with any servers and you would never get any results. I spent weeks on and off trying to figure this out. Was to told it was the alerting service but support and others were wrong.
Once that is setup you should be all set. Here is our reboot breakdown in triggers.
Hope this helps someone