One AX Restore to go!

The key to learning new things in life is “A Need” but the success of learning depends on “Desire” and “Passion”.

One of the challenges I was faced with recently was to restore the data from one AX environment to another environment. Hmm ok i agree it is not a challenge but rather a (ohhh) boring mundane task. By restore i mean, say you did some demos and testing and built up a lot of data and now you want to erase this environment so you can start a new cycle of testing. However you want to restore this data in a different environment, say from Staging to QA, so you can keep the testing scenarios you created. To handle this scenario what you would do is you just restore the business/OLTP database (MicrosoftDynamicAX) into a different environment. Ideally you should restore the Model database as well.

This activity involves a lot of things to be taken care of
- We want to preserve the access and rights of users who had access to the target environment(QA)
- We want to preserve all the Parameters in AX in target environment like Batch Server config, Help System URL, EP URL, Analysis Server URL, Report Server parameters etc.
- We would like the database restore to keep the target database’s Physical File Names and the logical names of the database files. eg. your Staging AX DB was named MicrosoftDynamicsAX_Stag with physical file names D:\MSSQL\Data\MicrosoftDynamicsAX_Stag.mdf, D:\MSSQL\Data\MicrosoftDynamicsAX_Stag.ldf and the QA was named MicrosoftDynamicsAX_QA with physical file names D:\MSSQL\Data\MicrosoftDynamicsAX_QA.mdf, D:\MSSQL\Data\MicrosoftDynamicsAX_QA.ldf. When you restore from a backup of Staging db the file names will now change. So during restore you have to change file names and after restore you have to change the Logical Name of the data files as well.

Why do something manually when we can automate it right? So i set about experimenting with PowerShell and you will find below the files that allow you to do that. All of this is parameter driven.

The PowerShell script will do the following
- Loads parameters from Parameters.txt and validates all the required parameters are present and have values
- Loads both AX and SQL PS Modules
- Exports all users and their roles in the current database and generates a PowerShell script file to recreate the users after restore
- Exports all the AX config parameters to local variables
- Extracts the current database’s physical file path and logical names of the data and log files
- Backups the existing database
- Stops AOS
- Kills all active connections to the database
- Restores the database from backup while preserving file names
- Renames the logical names of the data and log files
- Restarts AOS
- Restores AX config parameters
- Logs all activity
- Works in two modes Read Only and Read Write. When property ReadOnlyMode is set to anything other than False the script will only export users, export AX parameters, backup database, stop and start AOS. If we set it to True then in addition it will also restore db and update ax parameters.

There are few limitations of this script (which i am trying to fix)
- You have to run the script from AOS Server machine and with Admin rights (of course)
- The backup location for SQL database and the backup file to restore from, have to be local to SQL Server. Let us say your SQL Server is not on AOS Server then the backup path you specify say is D:\backup then this D drive and that folder has to exist on the SQL Server, although you are running the script from AOS Server.
- This assumes both the destination database and the source database have only one mdf and one ldf file each.
- Users exported are all assumed to be Windows users. Please feel free to update this code as you feel fit, there was no such need for me so i have left it as such.
- Updating especially Batch Server parameter in AX might fail because of duplicate rows, i am working on a fix for this

There were numerous issues that i had to resolve to get this work. And some of the most painful ones are listed below. You will understand these as you read the script
- Stop-Service/Start-Service might not work properly depending upon if you are doing this in a demo environment. In a demo environment (say Contoso) the connection between the computer and DC might not be proper and could cause these commandlets to fail.
- While restoring database the restore object’s NoRecovery property has to be set to False else your restore will just pure and misereably fail.
- During restore you need to specify the relocate files but only change the physical file path and not the logical file name.
- Logical file renaming is done later in the code after restore
- You could see issues in exporting users/roles but those can be safely ignored because the type of those users being exported are not windows users

The sources files have been inserted into this word document
SourceFiles

Main PS1 file

function validateSingleParameter($nameOfParameter)
{
    $retValueInner = 1;
    if(!$Global:Parameters.ContainsKey($nameOfParameter))
    {
        "VALIDATION FAILED::Parameter #$nameOfParameter# does not exist in the Parameters.txt file" | Out-File -FilePath $log -Append;
        $retValueInner = 0;
    }
    elseif($Global:Parameters[$nameOfParameter] -eq "")
    {
        "VALIDATION FAILED::Value for Parameter #$nameOfParameter# has not been specified in the Parameters.txt file" | Out-File -FilePath $log -Append;
        $retValueInner = 0;
    }
    return $retValueInner;
}

function validateParameters
{
    $retValue = 1;
    $retrievedValue = 0;
    
    $retrievedValue = validateSingleParameter "OutputFilePath"
    if($retValue -eq 1)
    {
        $retValue = $retrievedValue;
    }
    
    $retrievedValue = validateSingleParameter "ServerInstance"
    if($retValue -eq 1)
    {
        $retValue = $retrievedValue;
    }
    
    $retrievedValue = validateSingleParameter "Database"
    if($retValue -eq 1)
    {
        $retValue = $retrievedValue;
    }
    
    $retrievedValue = validateSingleParameter "BackupFilePath"
    if($retValue -eq 1)
    {
        $retValue = $retrievedValue;
    }
    
    $retrievedValue = validateSingleParameter "BackupFileToRestore"
    if($retValue -eq 1)
    {
        $retValue = $retrievedValue;
    }
    
    $retrievedValue = validateSingleParameter "AOSServer"
    if($retValue -eq 1)
    {
        $retValue = $retrievedValue;
    }
    
    $retrievedValue = validateSingleParameter "AOSInstance"
    if($retValue -eq 1)
    {
        $retValue = $retrievedValue;
    }
    
    $retrievedValue = validateSingleParameter "ReadOnlyMode"
    if($retValue -eq 1)
    {
        $retValue = $retrievedValue;
    }
    return $retValue;
}

#Date formatted as string to create a new folder
$dateCovertedToText = "{0:ddMMMyyyy_hhmmtt}" -f (Get-Date)

$currentWorkingDirectory = $MyInvocation.MyCommand.Path.Substring(0, $MyInvocation.MyCommand.Path.LastIndexOf("\"));
$log = $currentWorkingDirectory + "\Log_" + $dateCovertedToText + ".txt" 

#Create the log file (Overwrite if exists)
New-Item -Path $log -ItemType "File" -Force

#Load Parameters
& .\LoadParameters.ps1
"Loaded Parameters " | Out-File -FilePath $log

if(!(validateParameters))
{
    "ERROR::Parameters have not been properly defined. Stopping execution of script." | Out-File -FilePath $log -Append
    return
}

$readOnlyMode = $Global:Parameters["ReadOnlyMode"]
$server = $Global:Parameters["ServerInstance"]
$databaseName = $Global:Parameters["Database"]
$backupFileToRestore = $Global:Parameters["BackupFileToRestore"]

#Load Dynamics AX Module
& .\LoadDynamicsAXPSModule.ps1
"Loadeded AX Module " | Out-File -FilePath $log -Append

#Load AX variables
$aosServer = $Global:Parameters["AOSServer"]
$aosInstance = $Global:Parameters["AOSInstance"]

$axBatchServer = $Global:Parameters["AXBatchServer"]
$axHelpSystem = $Global:Parameters["AXHelpSystem"]
$axAnalysisServer = $Global:Parameters["AXAnalysisServer"]
$axReportServer = $Global:Parameters["AXReportServer"]
$axEPUrl = $Global:Parameters["AXEPUrl"]

#Export all users/roles and generate a ps1 file
#--------------------------------------------------
$filePath = $Global:Parameters["OutputFilePath"]
$lineToWrite = ""

New-Item $filePath -ItemType file -Force
$userList = Get-AXUser

$lineToWrite = "& .\LoadDynamicsAXPSModule.ps1"
$lineToWrite | Out-File -FilePath $filePath -Append

$lineToWrite = "& .\LoadSQLServerPSModule.ps1"
$lineToWrite | Out-File -FilePath $filePath -Append
foreach($user in $userList)
{
    if($user.AXUserId -ne "Admin")
    {
        $lineToWrite = "New-AXUser -AccountType " + $user.AccountType + " -AXUserId " + $user.AXUserId + " -Company " + $user.Company + " -UserDomain " + $user.UserDomain + " -UserName " + $user.UserName
        $lineToWrite | Out-File -FilePath $filePath -Append;

        $securityRoles = Get-AXSecurityRole -AxUserID $user.AXUserId
        foreach($securityRole in $securityRoles)
        {
            $lineToWrite = "Add-AXSecurityRoleMember -AOTName " + $securityRole.AOTName + " -AxUserID " + $user.AxUserID
            $lineToWrite | Out-File -FilePath $filePath -Append;
        }
    }
}
"Exported Users and roles to $filePath " | Out-File -FilePath $log -Append
#=====================================================
#Stop AOS
Stop-Service $aosInstance -WarningAction SilentlyContinue
"Stopped AOS instance $asoInstance on server $aosServer " | Out-File -FilePath $log -Append

#Load SQLPS Module
& .\LoadSQLServerPSModule.ps1
"Loaded SQL PS Module " | Out-File -FilePath $log -Append

#Save default parameters from the AX database
$axDBConn = New-Object System.Data.SqlClient.SqlConnection("Data Source=$server; Initial Catalog=$databaseName; Integrated Security=SSPI")
$axDBConn.Open();

#Load Batch Server parameter
$cmd = $axDBConn.CreateCommand()
$cmd.CommandText = "Select * From SYSSERVERCONFIG"
$result = $cmd.ExecuteReader();
$dataTable = New-Object System.Data.DataTable
$dataTable.Load($result)
foreach($dataRow in $dataTable.Rows)
{
    $axBatchServer = $dataRow.SERVERID;
}
"Batch Server = #$axBatchServer#" | Out-File -FilePath $log -Append

#Load Help System parameter
$cmd = $axDBConn.CreateCommand()
$cmd.CommandText = "Select * From SYSGLOBALCONFIGURATION Where NAME = 'HelpServerLocation'"
$result = $cmd.ExecuteReader();
$dataTable = New-Object System.Data.DataTable
$dataTable.Load($result)
foreach($dataRow in $dataTable.Rows)
{
    $axHelpSystem = $dataRow.VALUE;
}
"Help System parameter = #$axHelpSystem#" | Out-File -FilePath $log -Append

#Load Analysis Server parameter
$cmd = $axDBConn.CreateCommand()
$cmd.CommandText = "Select * From BIANALYSISSERVER Where ISDEFAULT = 1"
$result = $cmd.ExecuteReader();
$dataTable = New-Object System.Data.DataTable
$dataTable.Load($result)
foreach($dataRow in $dataTable.Rows)
{
    $axAnalysisServer = $dataRow.SERVERNAME;
}
"Analysis Server parameter = #$axAnalysisServer#" | Out-File -FilePath $log -Append

#Load Report Server parameters
$cmd = $axDBConn.CreateCommand()
$cmd.CommandText = "Select * From SRSSERVERS"
$result = $cmd.ExecuteReader();
$dataTable = New-Object System.Data.DataTable
$dataTable.Load($result)
foreach($dataRow in $dataTable.Rows)
{
    $axSRS_AOSID = $dataRow.AOSID;
    $axSRS_SERVERID = $dataRow.SERVERID;
    $axSRS_SERVERURL = $dataRow.SERVERURL;
    $axSRS_AXAPTAREPORTFOLDER = $dataRow.AXAPTAREPORTFOLDER;
    $axSRS_REPORTMANAGERURL = $dataRow.REPORTMANAGERURL;
    $axSRS_SERVERINSTANCE = $dataRow.SERVERINSTANCE;
    $axSRS_CONFIGURATIONID = $dataRow.CONFIGURATIONID;
}
"Report Server Parameter AOS Id = #$axSRS_AOSID#" | Out-File -FilePath $log -Append
"Report Server Parameter Server Id = #$axSRS_SERVERID#" | Out-File -FilePath $log -Append
"Report Server Parameter Server Url = #$axSRS_SERVERURL#" | Out-File -FilePath $log -Append
"Report Server Parameter Report Folder = #$axSRS_AXAPTAREPORTFOLDER#" | Out-File -FilePath $log -Append
"Report Server Parameter Report Manager Url = #$axSRS_REPORTMANAGERURL#" | Out-File -FilePath $log -Append
"Report Server Parameter Server Instance = #$axSRS_SERVERINSTANCE#" | Out-File -FilePath $log -Append
"Report Server Parameter Configuration Id= #$axSRS_CONFIGURATIONID#" | Out-File -FilePath $log -Append

#Load EP Parameter
$cmd = $axDBConn.CreateCommand()
$cmd.CommandText = "Select * From EPGLOBALPARAMETERS"
$result = $cmd.ExecuteReader();
$dataTable = New-Object System.Data.DataTable
$dataTable.Load($result)
foreach($dataRow in $dataTable.Rows)
{
    $axEPUrl = $dataRow.SEARCHSERVERURL;
}
"EP Parameter = #$axEPUrl#" | Out-File -FilePath $log -Append

$axDBConn.Close()

$sqlSrv = New-Object Microsoft.SqlServer.Management.Smo.Server($server);
$databaseObject = New-Object Microsoft.SqlServer.Management.Smo.Database;
$databaseObject = $sqlSrv.Databases.Item($databaseName);

foreach($fileGroup in $databaseObject.FileGroups)
{
    foreach($dataFile in $fileGroup.files)
    {
        $targetMDF = $dataFile.Name;
        $targetMDFFileName = $dataFile.FileName
        break;
    }
    break;
}

foreach($logFile in $databaseObject.LogFiles)
{
    $targetLDF = $logFile.Name;
    $targetLDFFileName = $logFile.FileName
    break;
}

#Load SQL PS Module changes the directory to SQLSERVER so storing the current path and switching it back after loading the SQL Powershell module
cd $currentWorkingDirectory 

#Backup folder path
#$sqlDBBackupPath =  $Global:Parameters["BackupFilePath"] + "\" + $dateCovertedToText + "\" 
$sqlDBBackupPath =  $Global:Parameters["BackupFilePath"] + "\" 

#Backup file name with full path
#$sqlDBBackupFile =  $sqlDBBackupPath + $Global:Parameters["Database"] + ".bak"
$sqlDBBackupFile =  $sqlDBBackupPath + $Global:Parameters["Database"] + ".bak"

#Switch to SQL Server directory to get to your database. Probably could optimize this code, but just want to get this done for now
Set-Location SQLSERVER:\SQL\$server\DEFAULT\DATABASES;

#Create the backup folder path
#New-Item -Path $sqlDBBackupPath -ItemType directory
"Created a new folder for backup " | Out-File -FilePath $log -Append

#Backup the database before we do a restore
Backup-SqlDatabase -ServerInstance $Global:Parameters["ServerInstance"] -Database $Global:Parameters["Database"] -BackupFile $sqlDBBackupFile
"Database backed-up to $sqlDBBackupFile " | Out-File -FilePath $log -Append

$sqlSrv = New-Object Microsoft.SqlServer.Management.Smo.Server($server);
$databaseObject = New-Object Microsoft.SqlServer.Management.Smo.Database;
$databaseObject = $sqlSrv.Databases.Item($databaseName);

$restoreDB = New-Object Microsoft.SqlServer.Management.Smo.Restore;

#The NoRecovery when set to true results in the database status going into a Restoring mode, not good.
$restoreDB.NoRecovery = $false;
$restoreDB.ReplaceDatabase = $true;
$restoreDB.Action = "Database"
$restoreDB.PercentCompleteNotification = 5;

#$restoreDB.Devices.AddDevice($backupFileToRestore, [Microsoft.SqlServer.Management.Smo.DeviceType]::File)
$backupDevice = New-Object Microsoft.SqlServer.Management.Smo.BackupDeviceItem($backupFileToRestore, "File");
$restoreDB.Devices.Add($backupDevice);

#Selects the first item in the backup set
$restoreDB.FileNumber = 1

$restoreDetails = $restoreDB.ReadBackupHeader($sqlSrv);
$relocateData = New-Object Microsoft.SqlServer.Management.Smo.RelocateFile
$relocateLog = New-Object Microsoft.SqlServer.Management.Smo.RelocateFile
$relocateData.LogicalFileName = $targetMDF
$relocateLog.LogicalFileName = $targetLDF
foreach($restoreFileItem in $restoreDB.ReadFileList($sqlSrv))
{
    switch($restoreFileItem.Type)
    {
        "D"{ $relocateData.LogicalFileName = $restoreFileItem.LogicalName; }
        default { $relocateLog.LogicalFileName = $restoreFileItem.LogicalName; }
    }
    
}
$relocateData.PhysicalFileName = $targetMDFFileName
$relocateLog.PhysicalFileName = $targetLDFFileName
$restoreDB.RelocateFiles.Add($relocateData);
$restoreDB.RelocateFiles.Add($relocateLog);
$restoreDB.Database = $databaseName

if($readOnlyMode -eq "False")
{
    #Terminate all connections to the SQL Database
    $sqlSrv.KillAllProcesses($databaseName);
}

if($readOnlyMode -eq "False")
{
    #Restore the database
    $restoreDB.SqlRestore($sqlSrv);
    "Database restored from backup file " | Out-File -FilePath $log -Append
}

if($readOnlyMode -eq "False")
{
    #We are going to modify the Logical name of the database and log file
    $sqlSrv.KillAllProcesses($databaseName);
    #Refresh the information in our SQL Server object
    $sqlSrv.Refresh();
}

#Reset the SQL Server object and the database object so that we release all connections
#Unable to change logical name of the database file if these objects are not re-initialized
$sqlSrv = New-Object Microsoft.SqlServer.Management.Smo.Server($server);
$databaseObject = New-Object Microsoft.SqlServer.Management.Smo.Database;
$databaseObject = $sqlSrv.Databases.Item($databaseName);
$databaseObject_PostRestore = New-Object Microsoft.SqlServer.Management.Smo.Database;
$databaseObject_PostRestore = $sqlSrv.Databases.Item($databaseName);

foreach($fileGroup in $databaseObject_PostRestore.FileGroups)
{
    foreach($dataFile in $fileGroup.files)
    {
        if($dataFile.Name -ne $targetMDF)
        {
            if($readOnlyMode -eq "False")
            {
                $dataFile.Rename($targetMDF);
            }
        }
        break;
    }
    break;
}

foreach($logFile in $databaseObject_PostRestore.LogFiles)
{
    if($logFile.Name -ne $targetLDF)
    {
        if($readOnlyMode -eq "False")
        {
            $logFile.Rename($targetLDF);
        }
    }
    break;
}

if($readOnlyMode -eq "False")
{
    $sqlSrv.KillAllProcesses($databaseName);
}
"Logical name of the database files has been changed " | Out-File -FilePath $log -Append

#Change working directory back to the current directory. 
#We need to do this because when we load the SQLPS module the directory is set to SQL Server
Set-Location $currentWorkingDirectory
"Changed working directory back to current working directory " | Out-File -FilePath $log -Append

#Start AOS
Start-Service $aosInstance -WarningAction SilentlyContinue
"Started AOS instance $asoInstance on server $aosServer " | Out-File -FilePath $log -Append

#We will now start updating parameters in Dynamics AX database
$axDBConn = New-Object System.Data.SqlClient.SqlConnection("Data Source=$server; Initial Catalog=$databaseName; Integrated Security=SSPI")
$axDBConn.Open();

$cmd = $axDBConn.CreateCommand()

if($readOnlyMode -eq "False")
{
    #Update batch server record in Dynamics AX database
    $cmd.CommandText = "Update Top(1) SYSSERVERCONFIG Set SERVERID='$axBatchServer', ENABLEBATCH=1"
    $cmd.ExecuteNonQuery()

    #Update Help system parameter in Dynamics AX database
    $cmd.CommandText = "Update Top(1) SYSGLOBALCONFIGURATION Set [VALUE]='$axHelpSystem' Where NAME = 'HelpServerLocation'"
    $cmd.ExecuteNonQuery()

    #Update Analysis Server parameter in Dynamics AX database
    $cmd.CommandText = "Update Top(1) BIANALYSISSERVER Set [SERVERNAME]='$axAnalysisServer' Where ISDEFAULT = 1"
    $cmd.ExecuteNonQuery()

    #Update Report Server parameter in Dynamics AX database
    $cmd.CommandText = "Update Top(1) SRSSERVERS Set [AOSID]='$axSRS_AOSID', [SERVERID] = '$axSRS_SERVERID', [SERVERURL] = '$axSRS_SERVERURL', [AXAPTAREPORTFOLDER] = '$axSRS_AXAPTAREPORTFOLDER', [REPORTMANAGERURL] = '$axSRS_REPORTMANAGERURL', [SERVERINSTANCE] = '$axSRS_SERVERINSTANCE', [CONFIGURATIONID] = '$axSRS_CONFIGURATIONID'"
    $cmd.ExecuteNonQuery()

    #Update EP Portal parameter in Dynamics AX database
    $cmd.CommandText = "Update Top(1) EPGLOBALPARAMETERS Set [SEARCHSERVERURL]='$axEPUrl'"
    $cmd.ExecuteNonQuery()
}

#Completed all our database operations and closing connection
$axDBConn.Close()