Getting Logi Info Studio To Play Nice With Git & CI/CD

A look at some basic limitations with Logi Analytics' IDE

Logi + Git

In recent weeks I’ve become intimately acquainted with Logi Analytics' IDE, called Logi Info Studio. I haven’t yet used it enough to have a strong sense of Logi’s platform as a whole, but there are definitely two aspects of developing with Info Studio which I’ve found outmoded: it is neither terribly source control friendly nor very well suited for continuous integration/deployment environments. That’s not to say it’s impossible to version control Logi applications or integrate them into CI/CD environments, but in my estimation it’s needlessly difficult. Below I’ll describe the limitations I’ve encountered and how I worked around them. Hopefully, Logi Analytics will address these issues in a future release of their IDE.

When you create a new application with Logi Studio, it produces around 200MB of static “engine” files that are required for the application to run. These files evidentially do not change over the life of the application, until you upgrade to a newer version of Info Studio. At that point some subset of the files must be regenerated. These facts alone indicate to me that engine files should not be source controlled. Logi Analytics seems to agree. In this blog post, they point out that over time source controlling engine files can cause the size of a git repository to balloon into the gigabyte range. They themselves do not include engine files in their repos. Logi Info Studio even has a relatively new ‘Source Code Repository Friendly’ option which - among other things - creates a .gitignore file that excludes Logi engine files and other autogenerated artifacts. Great! So I should be able to create a Logi application, ensure the ‘Source Code Repository Friendly’ option is checked, and life is good - right? Not quite.

Let’s say I create a new Logi application and check it in to source control. I should then be able to move to another computer, clone the repository, and open the application in Logi Info Studio. Instead, I get an error message:

Logi Error

Of course, this is because the engine files are missing. Logi does not directly address this issue in the blog post I cited earlier. However, they do suggest a process for deploying a Logi application. I can only presume this is also what they’d suggest to rectify the error I just described. This process basically amounts to creating a new application, then using git to overwrite customized files from source control. It feels like a hack. And it does not bode well for continuous integration and deployment environments.

In Logi Info Studio, use the New Application wizard to create an application in the desired deployment folder, such as c:\inetpub\wwwroot.

Proceed through the first two steps of the wizard, registering your new app with the web server, and then click Cancel at the wizard’s “Theme Selection” step.
In Git, use init to initialize the new application folder, such as c:\inetpub\wwwroot\yourLogiApp, as a new local repository.
In Git, add the remote repository server’s SSH/HTTPS address.
In Git, fetch the contents of the remote repository into the new local repository.
In Git, reset (–hard) to the desired branch (likely origin/master).

Like everyone else, my perception of How Things Ought To Work(tm) is skewed by past experiences. As a .NET developer, I’m accustomed to development projects being defined by one or more text files that are opened by my IDE (Visual Studio). Once loaded, any missing dependencies are pulled down via a package manager. I really have to wonder why Logi Info Studio doesn’t operate similarly, instead of defining an application by a very specific directory and file structure. If engine files are missing at runtime - simply build them, copy them, or do whatever Logi Info currently does when creating a new application. It would make things so much easier! Instead, any missing files or directories causes Info Studio to throw its arms in the air and say ‘Sorry dude! No idea what yer missing, but something ain’t right.'

A Workaround

I wanted a process whereby any developer at my company could clone the repo containing our Logi application, open it in Logi Info Studio, and immediately start working - much like they would with any Visual Studio based project. Recognizing that things would probably not end up quite that simple, I set my sights on making it as frictionless as possible. The first thing I did was update the .gitignore for the repo:

# Default Logi Analytics exclusions
*.lic
*.lgp
TeamDevelopment/
_rdLivePreview*.lgx
rdDataCache/
rdDownload/
rdErrorLog/

# Custom Logi Analytics exclusions
bin/
rdTemplate/

As indicated, that first section is from Logi’s default .gitignore. I added the latter two exclusions based on Logi’s blog post which says, “…the Logi Info ‘engine files’…are in the standard Logi app folders bin and rdTemplate…” (Emphasis mine.) These two directories comprise the bulk of a fresh Logi application’s size. Excluding them from your git repo automatically results in faster cloning, fetching, branching, and pushing.

Next I created a new repository called LogiBuildEngine, specifically and exclusively for - surprise! - Logi engine files. I then created a new Logi application and copied the generated bin and rdTemplate directories to the new repo. I also included two additional empty directories that I’ll talk about in a second. I ended up with something like this:

LogiBuildEngine
      \- bin
          \- [lots of files]
      \- rdDataCache
          \- .gitkeep
      \- rdDownload
          \- .gitkeep
      \- rdTemplate
          \- [lots of files]

Logi applications include two directories named rdDataCache and rdDownload, both of which are initially empty. The contents of these directories - once generated - should not be source controlled, as evidenced by Logi’s default .gitignore cited earlier. However, these directories must exist in order to load the application in Logi Info Studio. This poses a minor problem because as you may or may not know, git disregards empty directories. A common workaround for situations such as this is to add a single, empty file named .gitkeep. Really, it could be named anything, but I like this convention; it effectively conveys that the file only exists so git doesn’t ignore the directory. At this point, my LogiBuildEngine repo contained all the files and folders required by my Logi application which are excluded from the core application’s repository.

Now I just needed an automated way to consolidate these two repos together. For that, I wrote a PowerShell script named ClickToBuild.ps1 and placed it in the root directory for the core Logi application. The script either clones or updates the LogiBuildEngine repo, copies the engine files to the appropriate location in the core app directory, then finally copies the local Logi Info Studio license file - if it exists - into the core app directory. (Logi Analytics requires a machine-specific license file for each computer that runs Info Studio.)

Write-Host "Getting Logi build engine files, please wait...`n" -ForegroundColor Green

if (!(Test-Path ..\LogiBuildEngine))
{
  Set-Location ..
  git clone https://github.com/<COMPANY>/LogiBuildEngine.git
  Set-Location .\LogiBuildEngine
}
else
{
  Set-Location ..\LogiBuildEngine
  git fetch
  git checkout master -f
  git reset --hard origin/master
}

if ($LastExitCode -ne 0) {
  Write-Host "`nSomething went wrong :("
  exit
}

xcopy .\LogiApplicationBase\* ..\<CORE_REPO>\<LOGI_APP_DIRECTORY> /y /s /f

Write-Host "`nCopying Logi license file to application directory...`n" -ForegroundColor Green
Set-Location $PSScriptRoot

if (!(Test-Path .\<LOGI_APP_DIRECTORY>\*.lic)) {
  $programFiles = [Environment]::GetFolderPath("ProgramFiles")
  $logiPath = Join-Path $programFiles "\LogiXML IES Dev\LogiStudio"
  $licenseFile = ls $logiPath -Filter "*.lic" -Force
    | Select-Object -First 1 -ExpandProperty FullName

  if ($licenseFile -ne $null) {
    Copy-Item -Path $licenseFile -Destination .\<LOGI_APP_DIRECTORY>\
  } else {
    Write-Host "`nNo Logi license file found!`n" -ForegroundColor Green
  }
}

Write-Host "`nDone!`n" -ForegroundColor Green

Ok! Now, any developer can clone our core Logi application repository, run the ClickToBuild.ps1 script, launch Logi Info Studio and immediately begin developing. That’s more like it! And when the time comes to upgrade Logi Info Studio to a newer version, all we need to do is update the engine files in the LogiBuildEngine repo and have each developer re-run the ClickToBuild.ps1 script.

Logi + CI/CD

Now that the source control dilemma has been resolved, we can move on to the next conundrum: Getting a Logi application to work within a continuous integration and deployment pipeline. Logi Analytics’ recommended deployment paradigm is decidedly manual. I initially held out hope that somehow or another Logi Info Studio could be automated with a script. Maybe I was missing that section of the documentation? Alas, my support request asking about such a possibility was met with this response:

Logi Info Studio does not have any built-in functionality that allows developers to auto-deploy their application(s). However, you may find third party software that allows you to auto-deploy files and set the Logi Info engine files as those to deploy. Unfortunately, we may not be able to support you if a future issue arises that revolve around your deployment method.

With that, any thought of dynamically generating a Logi application on the build server is out the window. Luckily, it didn’t matter. TeamCity made this a very easy problem to solve. All I had to do was add two VCS roots to my build configuration - one for my core app, the other for the build engine files - and add a VCS checkout rule to the latter so that the resulting directory structure on the TeamCity build agent matched Logi’s requirements.

One problem remained. The server hosting the deployed application needed a unique Logi Info Studio license. I decided to put this in the LogiBuildEngine repo as well, resulting in a slightly modified directory structure:

LogiBuildEngine
      \- LogiApplicationBase
             \- bin
                 \- [lots of files]
             \- rdDataCache
                 \- .gitkeep
             \- rdDownload
                 \- .gitkeep
             \- rdTemplate
                 \- [lots of files]
      \- ServerLicense
             \- [logi-license.lic]

I then changed the VCS checkout rules for the build engine repo such that everything under LogiApplicationBase was copied to the core Logi app directory and everything under ServerLicense was copied to the root directory. After that, it’s all systems normal: run tests, create a Octopus deploy package, and push out the release as normal.