Connect Windows Server To Github

6 minute read

Description:

Here are the steps I did to connect my Jenkins Windows servers to Github. For now, instead of integrating Github directly into Jenkins, I installed Git on each server and scripted a way for them to pull down just their specific folder using sparse checkout ad hoc when the job is ran. While Jenkins can probably do this natively, I haven’t got there yet.

This is part of a 3 part series:

Update: It looks like if I wanted, I could set Jenkins to use sparse-checkout by just clicking Additional Options => Sparse Checkout Paths => Enter path and then entering . "$ENV:WORKSPACE\script name.ps1" as the build step. I will continue to use what I have for now but keep this mind for my next Jenkins setup.

To Resolve:

  1. On each node (not needed on master server), install Git and connect to a private repo for your company’s Github (if applicable):

    • First, in Azure AD create a service account, have it bypass 2FA (Security => Conditional Access => Policy Name => Exclude)
    • Now, in Github, join that user to your organization.
    • Login as that user and go to Settings => Developer Settings => Personal Access tokens => Create one with rights “repo” (full control)
    • Authorize it for SSO by hitting the drop down menu next to the token. Very important!
    • On each server, download and install 64 bit git. Make sure to ‘Use Git and optional Unix tools from the Command Prompt’ during install
    • Open ps not as admin:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    
    cd ~./Documents
    mkdir test-git
    cd test-git
    git init .
    git config --global user.name "Service Windows Jenkins"
    git config --global user.email "svc_windowsJenkins@domain.com"   
    git remote add origin https://github.com/yourCompany/YourJenkinsRepo
    git config core.sparsecheckout true
    Write-Output "root/folder1/example-sub-folder" | out-file -encoding utf8 .\.git\info\sparse-checkout
    git pull origin master
    # enter user name and personal access token in the window that pops up
    git config --global credential.helper manager
    
    • We will use something like this in our jenkins jobs going forward. It’s really neat because instead of checking out a single jenkinsfile like I do with my previous install, I just have each server to a git pull but just for that specific job’s folder at run time!
  2. At this point in the walkthrough, you should have a service account in Github tied to your organization and your repo and you have a personal access token stored on each Windows node in the credential manager.

    • Ensure that you can see your token in credential manager:
    • RDP to each Windows server and run control /name Microsoft.CredentialManager and look for your github entry
    • If you have lots of different entries or bulk, just delete everything and do the Git steps again. Here is what to run to clear CredMan:
    1
    2
    3
    4
    
    cmd /c "cmdkey /list" | ForEach-Object {if ($_ -like "*Target:*")
    {
    cmdkey /del:($_ -replace " ", "" -replace "Target:", "")
    }}
    
  3. Using your account which should be admin in Github, create a repo ‘windows-jenkins’ as a private repo initialized with a README and set the owner to your organization.

    • Add the svc_windowsJenkins@domain.com from the previous post as a maintainer under Repo Settings => Manage Access.
  4. OK, now we will do the final step which is to setup a job that will create powershell scripts on whatever host it runs on. These scripts will pull down from github specifically tied to the job (so they don’t download the whole repo on every run), they will move to the directory and execute whatever the scripts purpose is.

    • Create job: Freestyle project named ‘powershell-test2’
    • No schedule, no nothing - just a simple powershell step under build that contains
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    133
    134
    135
    136
    137
    138
    139
    140
    141
    142
    143
    144
    145
    146
    
    # create powershell script
    
    $ScriptFile = "q:\jenkins\workspace\powershell-test2\script.ps1"
    
    $text = @'
    If ( Test-Path "q:\jenkins\workspace\powershell-test2\temp" )
    {
       Remove-Item "q:\jenkins\workspace\powershell-test2\temp" -Recurse -Force
       New-Item -ItemType Directory -Path "q:\jenkins\workspace\powershell-test2\temp" | Out-Null
    }
    Else
    {
       New-Item -ItemType Directory -Path "q:\jenkins\workspace\powershell-test2\temp" | Out-Null
    }
    Set-Location "q:\jenkins\workspace\powershell-test2\temp"
    git init . 
    git remote add origin https://github.com/yourCompany/YourJenkinsRepo
    git config core.sparsecheckout true
    Write-Output "powershell-test" | out-file -encoding utf8 .\.git\info\sparse-checkout
    git pull origin master
    exit
    '@
    
    If ( -not ( Test-Path $ScriptFile))
    {
       New-Item -ItemType File -Path $ScriptFile -Value $text | Out-Null
    }
    Else
    {
       Remove-Item $ScriptFile -Force
       New-Item -ItemType File -Path $ScriptFile -Value $text | Out-Null
    }
    
    # create the second launcher
    
    $ScriptFile2 = "q:\jenkins\workspace\powershell-test2\script2.ps1"
    
    $text2 = @'
    $ScriptFile = "q:\jenkins\workspace\powershell-test2\temp\powershell-test\test.ps1"
    
    Function Get-ObfPass
    {
       $s = '76492d1116743f0423413b16050a5345MgB8AF'
       $s += 'IAMwA2AHYAQgAyAEEAcQB2AC8AZABnAFkAQgBL'
       $s += 'AHcAQQA9iHIARwBEAHcAPQA9AHwAMQA0ADcAZg'
       $s += 'AwAGQANAA5ADkAOQA1ADAAZgA2AGQAZQAyADEA'
       $s += 'YgAzAGUANQA89AGQAYwA3ADEAMwA1AGQAZQBmAG'
       $s += 'MAMQA2ADUANQA5ADkAYgBhAGYANAA2ADgAYgA4'
       $s += 'ADAAMAAyADAANwAyADIAYgAwADgANABjADgANg'
       $s += 'AyADcANwABBq2='
       $Key = Get-Content "q:\scripts\key.key"
       $user = 'blank'
       $pass = $s | ConvertTo-SecureString -Key $Key
       $cred = New-Object -Typename System.Management.Automation.PSCredential -Argumentlist $user, $pass
       $val = $cred.GetNetworkCredential().Password
       return $val
    }
    
    Write-Output "Running second script - inside"
    
    $Username = "domain.com\svc_windowsJenkins"
    $Password = Get-ObfPass
    $SecurePassword = ConvertTo-SecureString -String $Password -AsPlainText -Force
    $Credential = New-Object System.Management.Automation.PSCredential( $Username, $SecurePassword)
    
    $Start = @{ 
       'FilePath'         = 'powershell.exe'
       'Credential'       = $Credential
       'WorkingDirectory' = 'q:\jenkins\workspace\powershell-test2\temp\powershell-test'
       'ArgumentList'     = @( '-f', $ScriptFile,
          '-ExecutionPolicy', 'Bypass',
          '-NoProfile', '-Verb', 'RunAs'
       )
    }
    Start-Process @Start -Wait
    Write-Output "Running second script - inside - completed"
    
    exit
    '@
    
    If ( -not ( Test-Path $ScriptFile2))
    {
       New-Item -ItemType File -Path $ScriptFile2 -Value $text2 | Out-Null
    }
    Else
    {
       Remove-Item $ScriptFile -Force
       New-Item -ItemType File -Path $ScriptFile2 -Value $text2 | Out-Null
    }
    
    
    # launch the first script
    
    Function Get-ObfPass
    {
       $s = '76492d1116743f0423413b16050a5345MgB8AF'
       $s += 'IAMwA2AHYAQgAyAEEAcQB2AC8AZABnAFkAQgBL'
       $s += 'AHcAQQA9iHIARwBEAHcAPQA9AHwAMQA0ADcAZg'
       $s += 'AwAGQANAA5ADkAOQA1ADAAZgA2AGQAZQAyADEA'
       $s += 'YgAzAGUANQA89AGQAYwA3ADEAMwA1AGQAZQBmAG'
       $s += 'MAMQA2ADUANQA5ADkAYgBhAGYANAA2ADgAYgA4'
       $s += 'ADAAMAAyADAANwAyADIAYgAwADgANABjADgANg'
       $s += 'AyADcANwABBq2='
       $Key = Get-Content "q:\scripts\key.key"
       $user = 'blank'
       $pass = $s | ConvertTo-SecureString -Key $Key
       $cred = New-Object -Typename System.Management.Automation.PSCredential -Argumentlist $user, $pass
       $val = $cred.GetNetworkCredential().Password
       return $val
    }
    
    Write-Output "Running first script"
    
    $Username = "domain.com\svc_windowsJenkins"
    $Password = Get-ObfPass
    $SecurePassword = ConvertTo-SecureString -String $Password -AsPlainText -Force
    $Credential = New-Object System.Management.Automation.PSCredential( $Username, $SecurePassword)
    $Start = @{ 
       'FilePath'         = 'powershell.exe'
       'Credential'       = $Credential
       'NoNewWindow'      = $true
       'WorkingDirectory' = 'q:\jenkins\workspace'
       'ArgumentList'     = @( '-f', $ScriptFile,
          '-ExecutionPolicy', 'Bypass',
          '-NoProfile', '-Verb', 'RunAs'
       )
    }
    Start-Process @Start -Wait
    
    Write-Output "Running first script - Completed"
    Write-Output "Running second script"
    
    $Start2 = @{ 
       'FilePath'         = 'powershell.exe'
       'Credential'       = $Credential
       'NoNewWindow'      = $true
       'WorkingDirectory' = 'q:\jenkins\workspace'
       'ArgumentList'     = @( '-f', $ScriptFile2,
          '-ExecutionPolicy', 'Bypass',
          '-NoProfile', '-Verb', 'RunAs'
       )
    }
    Start-Process @Start2 -Wait
    Write-Output "Running second script - Completed"
    
    Write-Output "completed O_o"
    
    • Let me explain what this does:
      • At runtime, it will create two files under q:\jenkins\workspace\powershell-test2 called script.ps1 and script2.ps1
      • Since the nodes run as NT Authority\System, we have them launch a process as the service account that pulls down from Github (script.ps1)
        • As you can see, this is where that key file comes into play that we created in the previous post. Alternatively, you could try to have Jenkins inject the credential at run time.
      • We then do the same thing for script2.ps1 which will launch a script from the newly pulled down Github repo.
  5. Now that we have everything setup, the real power comes from the fact that from now on, you only have to use something like vscode to pull from github, make changes locally by git commit and then push the changes back to Github without even touching Jenkins or any of its nodes!

Comments