Pipeline actions in a single container (not isolated)


#1

Hi,

I’m just checking out Buddy after yesterday’s demo and I wanted to set up my first pipeline which is a “prepare” stage.

It has the following steps:

  • check the executable is present
  • check the version is equal or higher than my minimum version requirement
  • install required OS packages and libraries
  • check my package manager configuration for security vulnerabilities
  • run my package manager installation
  • execute lint checks
  • execute coding standards checks
  • execute unit tests
  • execute mutation tests

But it seems that each action is executed in isolation (each action spins up a new container), therefor the steps I execute to “prepare” my environment have no use.

Is there a way I can “keep” the container for the duration of this pipeline?


#2

Hey @Michelangelo,
You are right – each action is executed in an isolated container. The working dir of those actions is usually the pipeline’s filesystem, so that the dependencies pulled in one step are available for the next one.

Could you tell what do you mean by the steps you execute to prepare the environment? What are those steps exactly and what do you want to be available in the following actions?

Once we get that we’ll be to prepare a solution for you.


#3

Hi Kivlov,

My goal is best described in the following image of a common “build pipeline” I set up for projects.

Each pipeline consists of a series of actions where each action is responsible for one specific task.

So in Buddy, I started setting up my first pipeline in the same mindset

But since each action spins up a container, actions “Install required extensions” and “composer install” have no meaning since I need those extensions for my composer installation and execution of tasks in the actions following.

I could put all the actions in a single action, but that’s just messy and against the idea of pipelining and DevOps. Can I set in the pipeline configuration to use the same container to execute all the actions?


#4

Hey @Michelangelo,
Sorry to keep you waiting. Unfortunately, it’s not possible to use the same container in the following steps, because, as you noticed, every step is launched in a separate container.

Steps like fetching dependencies, however, can be freely done in separate actions, as each action is executed in the context of the pipeline’s filesystem. This means the dependencies will stay in the pipeline and will be available for subsequent actions.

As for the PHP extensions, you can install them in the Environment section of the PHP action (there’s a link under the command field). Buddy will install and cache them on the first execution which will reduce the execution time.

Summing up, the best idea is that you execute the first 4 steps from the screenshot in one action.

There is a better solution that you can apply, though: instead of doing checks in steps 1-2-3, you can build your own Docker image with the designated PHP version dependencies installed and use this image in the build action with composer install and the actions that follow.

We’ll be happy to assist should you need any help with that.


#5

Sure thing @Kivlov, I hope you had a great time at the conference :smiley:

I’m already looking at your suggested approach to spin up a container myself with the required extensions and libraries installed. Even though that works, it goes against the principle of DevOps where you should maintain a one-piece-flow approach.

Since the other time you mentioned that the filesystem is shared amongst each action, I’m looking at constructing the pipeline in such a way that each step will make use of this shared filesystem so I can maintain single responsibility in each action while the pipeline remains the value for the process.

Then I hit my second bottleneck (if I succeed in sharing the artefacts), a CI/CD process contains of multiple pipelines. Are these artefacts also shared amongst each of the pipelines?

Example:

  1. Pipeline for preparing the environment
  2. Pipeline for executing my fast tests
  3. Pipeline for setting up full infrastructure
  4. Pipeline for executing slow tests
  5. Pipeline for packaging and signing of the objects
  6. Pipeline for deploying to dev, staging and production

Even though they should all run in isolation, there are basics that are common amongst each action in each pipeline (e.g. PHP version, composer packages, NPM version, NPM packages, …) and it would be not making sense to have each pipeline to repeat the previous pipeline just to have everything available.


#6

It was great indeed, thanks! Regarding your post – filesystems are individual per pipeline and their content is not shared. You can, however, easily move files from one pipeline to another with the Copy Files action: