Thursday, July 13, 2017

Continuous Integration with PowerBuilder 2017, Bonobo Git and Jenkins

In a previous blog post I examined how we could use the Git MSSCCI provider from PB Software in order to use GitHub as a source code provider for PowerBuilder.

In this blog post we're going to take that to the next step, in that we're going to create a build machine separate from our PowerBuilder development machine and then set it up to perform continuous integration.




The term "continuous integration" has a somewhat ambiguous definition, but generally it means that:
  • developers check in changes frequently (at least daily) and
  • that build are done on a regular basis (at least daily, but can be as frequently as after each check-in)
Ideally, automated testing routines would be run on each build to ensure that feedback on any functionality that was broken by the latest code changes are returned to the developers as soon as possible.  Automated testing is outside the scope of this particular article

One of the new features added in PowerBuilder 2017 is a license free stand alone compiler and we're going to use that for this article.   If you are using a older version of PowerBuilder you could use the same approach using the command line argument feature of the PowerBuilder IDE, but it would require installing the full PowerBuilder IDE (including a license) on the build machine.  Alternatively, regardless of which version of PowerBuilder you're using you could use PowerGen in scripted mode.

Prerequisites

  • A windows machine (can be virtual) with .Net Framework 4.6 and IIS configured for ASP.Net
  • Bonobo Git Server
  • Jenkins
  • AutoCompile.exe from the C:\Program Files (x86)\Appeon\PowerBuilder 17.0\AutoCompiler directory from your PowerBuilder 2017 install

Install Bonobo Git Server

Bonobo is a ASP.Net application, hence the need for the machine we're installing it on to have the .Net Framework installed and IIS configured for ASP.Net.  Installation is fairly straightforward, as all you need to do is:

  • Copy the main folder from the unzipped download into the wwwroot folder for IIS
  • Give the IIS User modify and write permissions to the App_Data folder of the app
  • Convert the folder into an Application in IIS Manager
  • Ensure that the app pool that the application uses is based on .Net 4.0 rather than 2.0

Install Jenkins

The Jenkins install is also fairly straightforward as it uses a standard Windows installer.  After the install is complete it will generate a random password for the administrator into a file within the install and provide you with a message as to where to locate it.  You will need that password to do the initial login and configuration of Jenkins.

One you have started Jenkins it will prompt you as to the plugins you wish to install.  There are 1000+ plugins available for Jenkins.  The primary one we're interested in is the Git plugin, which is part of the default set of plugins that Jenkins will recommend for use. You can just accept their recommendations.

Install AutoCompile

Autocompile has a fairly simple install too, and automatically adds the install directory to the system path.

Configure Jenkins

Change the Jenkins URL from localhost to the actual name of the server that it's running on.


Create a Repository in Bononbo

You need to be logged in as the admin user to create a repository.  The only thing you have to provide is the name.


Once you're back at the list of repositories you can use the option there to copy the url for the new repository to the clipboard.


Configure the PowerBuilder IDE to use the new repository

Technically what you will be doing is configuring the PowerBuilder IDE to use a local repository and then configuring that local repository to use the new repository as a remote.  The detailed steps are outlined in my earlier article about using GitHub.  In summary, the steps are:


  1. Use TortoiseGit to create a new 'bare' repository in the directory where the PowerBuilder source is located.
  2. Create a small text file (e.g., readme.md) in the directory.
  3. Use TortoiseGit to add and commit the file.
  4. In PowerBuilder, select PGS Git MSSCCI as the source control system for the workspace.  Make sure that "Suppress prompts to overwrite read-only files" is checked.
  5. In PowerBuilder, add the application target(s) to source control.
  6. In PowerBuilder, add the remaining PowerBuilder objects to source control.
  7. In TortoiseGit settings for the directory, configure the Bonobo repository you created above as a remote repository for the local repository.
  8. In TortoiseGit, do a push from the local repository to the remote repository.

Create a new 'FreeStyle Project' in Jenkins



Under the Source Code Management section of the new project, select Git and then provide the repository URL and credentials to connect to the Git server.


Under the Build Triggers, specify any automatic build triggers you want to use.  For this example, I'm going to use "Poll SCM" and configure it to poll every 15 minutes.


Do a "Build Now" on the project


Because we haven't created any build steps yet this won't build anything yet.  What it will do is create a workspace under C:\Program Files (x86)\Jenkins\workspace and populate it with the current source code from the Git repository.

Create ORCAscript and batch files

Create an ORCAScript file to create PBLs from the source code (createpbls.orca):

 start session
 set debug true
 scc set connect property logfile "createpbls.log"
 scc connect offline  
 scc set target "pfc_ci\pfc_ci.pbt" importonly
 scc exclude liblist "pbdom\pbdom170.pbd"  
 scc refresh target 3pass
 scc close  
 end session  

Because Jenkins pulled the source code for us already we don't have to provide source control settings in the ORCAScript file and can use scc connect offline.  "ImportOnly" tells ORCAScript to build PBLs from source.  "Refresh Target 3pass" tells ORCAScript to do a multi-pass compile.  I'm using a sample app based on a recent version of the open source PFC, which now includes a reference to PBDOM.  Therefore I'm using "scc exclude liblist" to tell ORCAScript to ignore that library during the source code import.

Create a batch file that will run the ORCAScript executable on the ORCAScript file (run_orcascript.cmd).

 orcascr170 createpbls.orca  

Create a batch file to call the PowerBuilder stand alone compiler.  We're going to need to pass a number of arguments to the compiler.  Fortunately, the application project object in PowerBuilder shows you the arguments you would need to pass to match the settings in the project object.



Using those arguments - modified slightly because we're deploying in a different location - we should have something like this (run_pbc.cmd):

 pbc170 /d "pfc_ci\pfc_ci.pbt" /o "pfc_ci\pfc_ci.exe" /w n /m n /x 32 /p "PowerBuilder Enterprise Series" /cp "Appeon" /de "Appeon Product File" /v "1.0.0.1" /fv "1.0.0.1"   

Finally, create a batch file that will copy the generated exe and pbd files into another directory when the build is complete (copyfiles.cmd).

 md c:\builds\%JOB_NAME%_%BUILD_NUMBER%\  
 FOR /d %%a in (*) do copy %%a\*.exe c:\builds\%JOB_NAME%_%BUILD_NUMBER%\  
 FOR /d %%a in (*) do copy %%a\*.pbd c:\builds\%JOB_NAME%_%BUILD_NUMBER%\  

This batch files uses environment variables that Jenkins makes available when the batch file is run to create a separate directory for each build.

Create Build Steps

Go back into the Jenkins project and under Build add a Build Step that executes a batch command.


Specify the run_orcascript.cmd file as the first batch file to run.


Add another build step after the orcascript step and point this one at the run_pbc.cmd file.  Finally, create one more build step after the run_pbc one and have it run the copyfiles.cmd file.

Do a Build Now on the project

We're going to test out our scripts to make sure that Jenkins can do the build.  Once you've scheduled a build you should see it being processed in the list of builds.


If you click on the build, you'll be taken to another page with more details, including the ability to view the console output from the running build.


Test the SCM poll

Now that we know the build works, let's see if we actually have continuous integration.  Go back into PowerBuilder and check out an object.  Make a trivial change and commit it.  Then using TortoiseGit, push that change to the Bonoho repository.  Now watch Jenkins and what you should see is that some time after Bonoho has been updated (depending on what you set the SCM polling time to in Jenkins) Jenkins will automatically launch a build of the project.

Next Steps

That gives us the basics of a continuous integration setup.  I'm going to be looking at taking it a bit further.  In particular.

  • There is a Jenkins plugin for the Visual Studio command line build utility (MSBUILD).  I'm looking a creating a similar plugin for the PowerBuilder build utility.
  • Integration with JIRA.  Rather than firing a build on every checkin, the checkins would be tagged with a JIRA issue and the build would only fire when the JIRA issue is moved to the JIRA "Waiting for Deployment" status.





19 comments:

Unknown said...

Hi Bruce,

One question on this why do we need bonobo git server for this? can we do it without installing the bonobo git server and connect Jenkins directly to the Git Repository

Thanks,
Shaila Panwar

Bruce Armstrong said...

>>One question on this why do we need bonobo git server for this? can we do it without installing the
>>bonobo git server and connect Jenkins directly to the Git Repository

You need some git server. I just used Bonobo for this demonstration. You can use any Git server (including GitHub) that you want.

Unknown said...

Thanks for the input again..

What I am looking for is , if we have any way to connect Jenkins to Git Repository (Online Repository on git.com) without installing the Git Server.

I was trying to do that but unfortunately its not working.

Thanks,
Shaila

Unknown said...

Hi Bruce,

I managed to configure the Git repository via Jenkins but I am stuck on the step where exe has to be build i.e run_pbc.cmd.
I am using below arguments to run it
pb125 /d "benchmark\benchmark.pbt" /o "benchmark\benchmark.exe" /w n /m n /x 32 /p "PowerBuilder Enterprise Series" /cp "Sybase, Inc." /de "Sybase, Inc. Product File"
/v "1.0.0.1" /fv "1.0.0.1"

I am doing it via PB125 instead of PB170.
But Jenkins does not seems to be moving on this step it just keep loading and do not show anything.I could not see any failure as well on the console output of jenkins.
any suggestion on this ?

Thanks,
Shaila

Bruce Armstrong said...

>>I am doing it via PB125 instead of PB170.

The PBC170 that I'm using in the demonstration is a new feature of PowerBuilder that was introduced with 2017. PB 12.5 doesn't have that feature.

You would need to either:

a) Install the PowerBuilder IDE on the build machine and use it's command line compile feature

http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc00844.1252/html/pbug/BABDDFCJ.htm

b) Use PowerGen to do the compile.

https://ecrane.com/powergen-overview/

Unknown said...

Hi Bruce,

I tried to use the PB command line feature for the compilation but this does not seems to be working or may be I am doing something wrong here.
I tried with ORCA script then and it worked for me in Jenkins.
I am able to perform the build and open work space using CMD but it opens the PB only and then ask me to enter the information to run the EXE project.
Via Jenkins it keeps on loading and does not seems to go anywhere so I had to stop the build in Jenkins.

Do you have any sample script for the PB125 CMD for Jenkins.

Thanks,
Shaila


Bruce Armstrong said...

The command line arguments are documented in the help. Here's a link to the PBL Peeper changes to PowerBuilder that show the most important ones:

http://www.techno-kitten.com/Changes_to_PowerBuilder/New_In_PowerBuilder_8/PB8_New_-_Building_and_deployi/pb8_new_-_building_and_deployi.html

For example, assuming there is an application project in the this target, the following would launch the PowerBuilder IDE and deploy the app using that target.

pb125 /t pfc\generic_pfc_app.pbt /d /ou build.log

Unknown said...

Hi Bruce,

Do you know any tool to write test script for PowerBuilder and those can be used in Jenkins and can record the output.
I am looking to automate the PB testing for UI application written in PB.
Let me know in case you have any information on this.

Bruce Armstrong said...

That's a real good question for the Appeon Community: https://community.appeon.com/home

Unknown said...

Hi Bruce,

I was able to manage to create build using Jenkins based on this blog.
But suddenly it stopped working with the below error:
C:\Program Files (x86)\Jenkins\workspace\benchmark>run_orcascript.cmd

C:\Program Files (x86)\Jenkins\workspace\benchmark>orcascr125 createpblspbdexe.orca

Error checking out a SySAM license.
Sybase (R) OrcaScript interpreter version 12.5
Using ANSI source file createpblspbdexe.orca.
Start Session
Last Command Failed.

I have the Licensed version of PowerBuilder and I am able to run the same scripts outside of Jenkins but it does not working through Jenkins.
Any input on this will be helpful.

Thanks!!

Bruce Armstrong said...

All I can suggest is doing the standard Sysam license debugging.

Unknown said...

HI Bruce,

The is what I am getting in the log file:

CO Filter: 'PE=EE;LT=EV'
CO Feature: 'PowerBuilder'
CO Version: '2010.1225'
2018/04/02 17:23:31 Error (131281): Failed to obtain 1 license(s) for PowerBuilder feature from license file(s) or server(s).
2018/04/02 17:23:31 Error (131292): Licenses exist for pb125GA, but a license containing the configured (PE=EE;LT=EV) attributes could not be obtained. Verify that pb125GA is configured to use the correct type of license, and either reconfigure; or generate and deploy the desired license from the Sybase Product Download Center.
2018/04/02 17:23:31 Error (0): License feature name: PowerBuilder
2018/04/02 17:23:31 Error (0): License search path: C:\windows\system32\config\systemprofile\AppData\Local -
2018/04/02 17:23:31 Error (0): \Sybase\PowerBuilder 12.5\VS13847809_20171122_080436.lic
2018/04/02 17:23:31 Error (0): FLEXnet Licensing error:-73,125
2018/04/02 17:23:31 Error (0): For further information, refer to the Sybase Software Asset Management website at http://www.sybase.com/sysam

Failure in: sylapi_checkout() - code: 131145
Still waiting for the license initialization...
License has truly failed.

I am understanding here how I can configure the license to use the PB125GA.
Any suggestion?

Bruce Armstrong said...

This really is a question for the SAP https://www.sap.com/community/topic/powerbuilder.html) or Appeon (https://www.sap.com/community/topic/powerbuilder.html) community forums. It's not something I can debug through blob comments.

Ganesh said...

Hi Bruce,
We are setting up our PB project in git and I m trying to build using your jenkins tutorial.

I was wondering since with new Appeon Powerbuilder git support.Powerbuilder exports the source files to ws_objects folder however the target file doesnt get updated.

I m facing issue with orac script which builds pbls off the source files which are in ws_objects.

Please advise if there is any extra modifications I need to do to my orca script.

Thanks in Advance.

Ganesh

Aayush said...

Hi Bruce, I tried installing Orcascript plugin in jenkins. i am using powerbuilder 2019. The error i am getting on jenkins is that it says orcascr170.exe does not exists. i have given the path for orcascr190 but it is throwing error for 170.

Aayush said...

Hi Bruce, i am trying to build the pbl from jenkins. i am using powerbuilder 2019. i installed the orcascript plugin and pbc plugin. the auto compiler installation gave orcascr190 and pbc190. while building the job, i am getting error saying orcascr170 does not exists. can you help on this?

thanks

Bruce Armstrong said...

Aayush,

The plugin would need to be updated to call orcascr190 rather and orcascr170. The source code is available on the Github site.

Unknown said...

Hi Bruce,

Our company is looking to do CI with our PB applications and GitLab and Jenkins.

We would like to know if we can freely use your OrcaScript, and PBCompile plug-ins for our CI effort?

Thank you,

Tony

Bruce Armstrong said...

Absolutely, its open source. However, you may want to look at the new AutoBuild feature in PowerBuilder 2021. It pretty much eliminates the needs for the plugins. You can just call it directly from Jenkins:

https://docs.appeon.com/ps2021/Build_the_PowerServer_project_with_commands.html