Tuesday, July 12, 2011

Yet another trigger : installTrigger

At work I use CCNet for 2 main different things :
° as Continuous Integration : compile, test, package, code statistics, ...
° as Continuous Installation : install our software at the customers site ( see articles)

For the installation part, I previously changed the ccnet.config to reflect the needed changes. For example change the requested time of a schedule trigger, add the names of the server(s) where to install or not to install the software.
Now this works great, but it is error prone!
Remember ccnet.config is the core configuration of CCNet, a typo there could stop the service. And reviving it is not fun, remember I have a small 100 servers to maintain!!

So the idea came for a new trigger, to remove the parts that change a lot in ccnet.config to another file. Just moving to another file is not enough though, if there is an error in that file, ccnet may not crash. Meaning that using the pre-processor or XML-Entities is out of the question.

Now I already have an xml file for each customer, that holds what software the customer has, what sql server settings are needed, where the software needs to come, ....

Meaning this trigger should just read this xml file, and return the needed things.
I just needed to add the wanted integration time and wanted version to this file, to get it to work.


The new trigger is the InstallTrigger (cool name actually) and is a copy of the schedule trigger with the following extra parts :
° CheckInstallationNeededForProgramToInstall (the name of the program as known in the xml file)
° DeploySettingFilePath : where the xml file is located
° CruiseInstallProject : the corresponding CCNet project name that does the installation (more on this later)
° UpdateDeploySettingsProject : Name of CCNet project that downloads updated xml files from the ftp site
° removed the Time property, as this will come from the xml file


The most important things of the code :
° at start, get the wanted datetime of the xml file, if the date is before today, copy the time part to todays date
° when an integration is done, and the installation failed, get the wanted datetime from the xmlfile, but add 1 day (re-schedule automatically).
° when an integration is done, and the installation was ok, return datetime.maxvalue
° when the UpdateDeploySettingsProject ran, re-read the xml file.
° use datetime.utcNow iso dateTime.Now, this is a LOT faster (look here

Now why I added the CruiseInstallProject property?
One could easily say that this is not needed, because the trigger is inside the project that will install the requested software.
Wrong bet :-)
Remember a previous post : forcing multiple builds at once

So when I want to install, let's say Bookkeeping, also our security module (for login and so) needs to be installed.
So I have the following CCNet projects :
CCNetProjectName InstallTrigger property cruiseInstallProject Tasks
InstallSecurity InstallSecurity install security program
InstallBookKeeping install bookkeeping program
FullInstallBookkeeping InstallBookKeeping
  1. ForceBuild InstallSecurity
  2. ForceBuild InstallBookKeeping

This makes it a lot safer to do the installations, and even gives us an easy way to schedule the same program at different dates/times for different customers.
It is the same CCNet project, just the xml file for the involved customer needs to change.

Below is the code of the InstallTrigger.

Imports Exortech.NetReflector
Imports ThoughtWorks.CruiseControl.Remote
Imports ThoughtWorks.CruiseControl.Core.Config
Imports System.Globalization
Imports ThoughtWorks.CruiseControl.Core.Util

<ReflectorType("installTrigger")> _
Public Class InstallTrigger
Implements ITrigger

Private m_name As String
Private dtProvider As DateTimeProvider
Private m_nextBuild As DateTime = DateTime.MinValue
Private m_previousBuild As DateTime
Private triggered As Boolean
Private _includeServerNames As String()
Private _excludeServerNames As String()

Private _serverConfig As Data.DeploySetting = Nothing
Private _requestedInstallDate As DateTime = DateTime.MaxValue
Private _cruiseInstallProject As String = Nothing
Private _cruiseInstallProjectStateFile As String

Private _updateDeploySettingsStateFile As String
Private _UpdateDeploySettingsVersion As String
Private _UpdateDeploySettingsProject As String = "UpdateDeploySettings"
Private _UpdateDeploySettingsProjectLastTimeChecked As DateTime
Private _integrationDone As Boolean
Private _ServerNameIsOk As Boolean

Public Sub New()
Me.New(New DateTimeProvider())
End Sub

Public Sub New(ByVal dtProvider As DateTimeProvider)
Me.dtProvider = dtProvider
End Sub

<ReflectorArray("includeServerNames", required:=False)> _
Public Property IncludeServerNames() As String()
Get
Return _includeServerNames
End Get
Set(ByVal value As String())
_includeServerNames = value
End Set
End Property

<ReflectorArray("excludeServerNames", required:=False)> _
Public Property ExcludeServerNames() As String()
Get
Return _excludeServerNames
End Get
Set(ByVal value As String())
_excludeServerNames = value
End Set
End Property

<ReflectorProperty("name", Required:=False)> _
Public Property Name() As String
Get
If m_name Is Nothing Then
m_name = [GetType]().Name
End If
Return m_name
End Get
Set(ByVal value As String)
m_name = value
End Set
End Property

<ReflectorProperty("buildCondition", Required:=False)> _
Public BuildCondition As BuildCondition = BuildCondition.IfModificationExists

<ReflectorArray("weekDays", Required:=False)> _
Public WeekDays As DayOfWeek() = DirectCast(DayOfWeek.GetValues(GetType(DayOfWeek)), DayOfWeek())

Private _checkInstallationNeededForProgramToInstall As String
Private _deploySettingFilePath As String = "D:\InstallPath"

<ReflectorProperty("checkInstallationNeededForProgramToInstall", required:=True)> _
Public Property CheckInstallationNeededForProgramToInstall() As String
Get
Return Me._checkInstallationNeededForProgramToInstall
End Get
Set(ByVal value As String)
Me._checkInstallationNeededForProgramToInstall = value
End Set
End Property

<ReflectorProperty("deploySettingFilePath", required:=False)> _
Public Property DeploySettingFilePath() As String
Get
Return Me._deploySettingFilePath
End Get
Set(ByVal value As String)
Me._deploySettingFilePath = value
End Set
End Property

<ReflectorProperty("updateDeploySettingsProject", required:=True)> _
Public Property UpdateDeploySettingsProject() As String
Get
Return Me._UpdateDeploySettingsProject
End Get
Set(ByVal value As String)
Me._UpdateDeploySettingsProject = value
End Set
End Property


<ReflectorProperty("cruiseInstallProject", required:=True)> _
Public Property CruiseInstallProject As String
Get
Return _cruiseInstallProject
End Get
Set(value As String)
_cruiseInstallProject = value
End Set
End Property

Private Sub SetNextIntegrationDateTime()

Dim now As DateTime = dtProvider.Now

m_nextBuild = RequestedInstallDateTime()
If m_nextBuild = DateTime.MaxValue Then Return

If now >= m_nextBuild OrElse now.Date = m_previousBuild.Date Then
m_nextBuild = m_nextBuild.AddDays(1)
End If

m_nextBuild = CalculateNextIntegrationTime(m_nextBuild)
End Sub

Private Function CalculateNextIntegrationTime(ByVal nextIntegration As DateTime) As DateTime
While True

If IsValidWeekDay(nextIntegration.DayOfWeek) Then
Exit While
End If
nextIntegration = nextIntegration.AddDays(1)
End While
Return nextIntegration
End Function

Private Function IsValidWeekDay(ByVal nextIntegrationDay As DayOfWeek) As Boolean
Return Array.IndexOf(WeekDays, nextIntegrationDay) >= 0
End Function

Public Overridable Sub IntegrationCompleted() Implements ITrigger.IntegrationCompleted

Dim now As DateTime = dtProvider.Now

If triggered Then
m_previousBuild = now

'to force a re-read of the deploymentsettings file after X seconds.
'State file does not seem to be updated yet
_UpdateDeploySettingsProjectLastTimeChecked = DateTime.UtcNow
_integrationDone = True

End If
triggered = False
End Sub

Public ReadOnly Property NextBuild() As DateTime Implements ITrigger.NextBuild
Get
If m_nextBuild = DateTime.MinValue Then
'first time initialise
FirstTimeInitialize()

If _ServerNameIsOk Then
ThoughtWorks.CruiseControl.Core.Util.Log.Debug("Loading deploysettings file ")
_serverConfig = New Data.DeploySetting(DeploySettingFilePath)
SetNextIntegrationDateTime()
End If
End If

If Not _ServerNameIsOk Then Return DateTime.MaxValue

'to keep watching the deploy settings file after a second 'Fire'
If isDeploySettingsFileIsUpdated() Then
ThoughtWorks.CruiseControl.Core.Util.Log.Debug("Re-loading deploysettings file")
_serverConfig = New Data.DeploySetting(DeploySettingFilePath)
SetNextIntegrationDateTime()
End If

Return m_nextBuild
End Get
End Property

Public Function Fire() As IntegrationRequest Implements ITrigger.Fire
Dim now As DateTime = dtProvider.Now

If now > NextBuild AndAlso IsValidWeekDay(now.DayOfWeek) Then
triggered = True
Return New IntegrationRequest(BuildCondition, Name)
End If

Return Nothing
End Function

Private Function ServerNameIsOk() As Boolean
Dim includeServersSpecified As Boolean = IncludeServerNames IsNot Nothing AndAlso IncludeServerNames.Length > 0
Dim excludeServersSpecified As Boolean = ExcludeServerNames IsNot Nothing AndAlso ExcludeServerNames.Length > 0

If includeServersSpecified Then
For Each srv In IncludeServerNames
If srv.Equals(System.Net.Dns.GetHostName, StringComparison.CurrentCultureIgnoreCase) Then Return True
If srv.Equals(System.Environment.MachineName, StringComparison.CurrentCultureIgnoreCase) Then Return True
Next
Return False
End If

If excludeServersSpecified Then
For Each srv In ExcludeServerNames
If srv.Equals(System.Net.Dns.GetHostName, StringComparison.CurrentCultureIgnoreCase) Then Return False
If srv.Equals(System.Environment.MachineName, StringComparison.CurrentCultureIgnoreCase) Then Return False
Next
Return True
End If

Return True

End Function

Private Function RequestedInstallDateTime() As DateTime

Try
Dim CCNetInfo = GetInfoFromCCNetStateFile(_cruiseInstallProjectStateFile)
Dim LastSucessFullLabel = CCNetInfo.LastSucessFullLabel

Dim projectVersion As Version
If String.IsNullOrEmpty(LastSucessFullLabel) OrElse String.Equals("unknown", LastSucessFullLabel, StringComparison.CurrentCultureIgnoreCase) Then
projectVersion = New Version(0, 0)
Else
projectVersion = New Version(LastSucessFullLabel)
End If

'specific company code to get the data from the config, can not post this :-(
'dummy code folows
Dim progInfo = _serverConfig.GetInfo(CheckInstallationNeededForProgramToInstall)
'end dummy code

If projectVersion < progInfo.WantedVersion Then
If progInfo.WantedFrom.Date < Now.Date Then
'should the server be down for some time or so
_requestedInstallDate = Now.Date.AddHours(availableApplication.WantedFrom.Hour).AddMinutes(availableApplication.WantedFrom.Minute)
Else
_requestedInstallDate = progInfo.WantedFrom
End If

If _integrationDone Then
'if installation failed, re-schedule it for the next day at the same time, giving us a day to investigate,
'otherwise try again for temporary errors
If Not CCNetInfo.IsSuccesfull Then _requestedInstallDate = _requestedInstallDate.AddDays(1)

_integrationDone = False
End If
Else
_requestedInstallDate = DateTime.MaxValue
End If

Catch ex As Exception
'on error, just set it to not scheduled
_requestedInstallDate = DateTime.MaxValue
Dim ei = String.Format("Error in RequestedInstallDateTime : {0} ", ex.ToString)
ThoughtWorks.CruiseControl.Core.Util.Log.Error(ei)
EventLog.WriteEntry("CCNet-InstallTrigger", ei, EventLogEntryType.Error)
End Try

Return _requestedInstallDate

End Function

Private Function GetInfoFromCCNetStateFile(cruiseProjectStateFile As String) As StateFileInfo

Dim result As New StateFileInfo

Try
If IO.File.Exists(cruiseProjectStateFile) Then
Dim stateFileContents As String = IO.File.ReadAllText(cruiseProjectStateFile)

Dim xdoc As New Xml.XmlDocument
xdoc.LoadXml(stateFileContents)

result.LastSucessFullLabel = xdoc.SelectSingleNode("IntegrationResult/LastSuccessfulIntegrationLabel").InnerText

Dim LastIntegrationStatus = xdoc.SelectSingleNode("IntegrationResult/Status").InnerText

result.IsSuccesfull = LastIntegrationStatus = "Success"
End If
Catch ex As Exception
EventLog.WriteEntry("CCNet-InstallTrigger", "Error reading state file of project : " + cruiseProjectStateFile + ex.ToString, EventLogEntryType.Error)
End Try

Return result
End Function

Private Function isDeploySettingsFileIsUpdated() As Boolean

If DateTime.UtcNow.Subtract(_UpdateDeploySettingsProjectLastTimeChecked).TotalSeconds > 5 Then
'check the state file every X seconds

Dim news = GetInfoFromCCNetStateFile(_updateDeploySettingsStateFile)
_UpdateDeploySettingsProjectLastTimeChecked = DateTime.UtcNow

If _UpdateDeploySettingsVersion <> news.LastSucessFullLabel Then
_UpdateDeploySettingsVersion = news.LastSucessFullLabel
Return True
End If
End If

Return False
End Function

Private Sub FirstTimeInitialize()
_cruiseInstallProjectStateFile = IO.Path.Combine(New IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly.Location).DirectoryName, CruiseInstallProject + ".state")
_updateDeploySettingsStateFile = IO.Path.Combine(New IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly.Location).DirectoryName, _UpdateDeploySettingsProject + ".state")
_UpdateDeploySettingsVersion = GetInfoFromCCNetStateFile(_updateDeploySettingsStateFile).LastSucessFullLabel
_UpdateDeploySettingsProjectLastTimeChecked = DateTime.UtcNow

_ServerNameIsOk = ServerNameIsOk()
End Sub

Private Class StateFileInfo
Public Property LastSucessFullLabel As String
Public IsSuccesfull As Boolean
End Class

End Class