Concurrent Actions


#1

It would be great if there was a way to run certain pipeline actions at the same time, instead of waiting for each to finish.

My use case is that I have actions in my pipeline to upload my site assets to S3, and then upload my site code to my server. Those actions both take a little while, but there is no reason why they couldn’t be done at the same time for efficiency, before carrying on with the rest of the pipeline.

Currently, my pipeline looks like this:

  1. Build files with Gulp (~25 secs)
  2. Upload Assets to S3 (~30 secs)
  3. Upload Code to Server (~20-30 secs)
  4. Run SSH commands to send site live (~2 secs)
  5. Post Success to Slack (~2 secs)

Ideally, it would look like this:

  1. Build files with Gulp
  2. {Upload Assets to S3 +++ Upload Code to Server}
  3. Run SSH commands to send site live
  4. Post Success to Slack

ie, both upload actions can happen concurrently, but I don’t want the next action to run until both are complete.

Not sure if this is more a technical challenge, or more a UI challenge… but the challenge is there! :slight_smile:


#2

Hi Michael,

Thanks for detailed description. The challenge is already overcome - we call it concurrent [execution] runs.
In order to increase the number you’ll have to upgrade your plan. Drop a word to support@buddy.works and let us know how many projects and users you have and which plan you’re on and we’ll work something out.


#3

Hey @Kivlov - thanks for the answer, i will get in touch soon about upgrading our plan.

However - I was under the impression that the existing ‘concurrent execution runs’ feature meant that you could run multiple pipelines concurrently, as opposed to multiple actions within a pipeline?


#4

Your impression is right: concurrent executions refer to the pipelines being run in parallel. The actions are always executed step-by-step. We’ve always struggled to make the naming understandable for everybody, looks like there’s still place for improvement.


#5

Heh, fair enough. Going back to my original comment then - is there the possibility of concurrent actions within the pipeline, or is there another way to achieve that?


#6

Could you give an example of actual usage of such setup? I can’t think of anything than, say, running two builds in parallel, and that’s why we limited the concurrency in basic accounts - because it’s very resource consuming on our side.


#7

Only the example that I put in my original post. I get the reasoning behind limiting the parallel build processes because of the resources. I guess even though my example is concurrent ‘upload’ actions instead of ‘build’ actions, they would still require separate processes to run?


#8

@michaelroper, we’ve added it to our backlog.

We already have an idea how to do that. Still, if we are to prepare a solution for that, we need to know if gulp actions process assets and other files, or else, assets don’t have to be processed with gulp.
Let us know.


#9

Hello,

I can give you an example of a pipeline that needs concurrent actions.

Imagine typical PHP monolithic application that requires following steps during a build process:

  1. run “composer install” to get latest PHP dependencies
  2. run “npm install” or “yarn install” to get frontend dependencies
  3. run DB migrations
  4. clear runtime files from the backend
  5. run frontend build command
  6. start file synchronization

So in this pipeline, we can run 1 and 2 concurrently then run 3, 4 and 5 concurrently and in the end run 6.
But what would be really cool if we would be able to run a set of actions step-by-step concurrently with other sets of actions.

For example:
we can start 1 and 2, then after 1 is finished we can start 3 while 2 still working. Then after 1 and 3 are finished we can start 4 and when 2 is finished we can start 5 concurrently with the first set of actions and then start 6 only once everything else is finished.

More visual representation:

composer install -------------------------- “npm install”
run DB migrations -------------------------- run frontend bulid command
clear runtime files --------------------------

                        start file synchronization

Concurrent actions is a must-have feature because any application that needs CI/CD tool like Buddy would have a lot of actions that should be done concurrently. And without concurrent actions, builds would take a lot more time that they should.

I’m analyzing Buddy to decide whether our team should purchase a subscription or not but lack of concurrent actions really surprised me in a bad way.

Hope you would add concurrent actions support in the nearest future.

I can give you more examples and even mockups of interface if you would like to.

With best regards,
Dmitry


#10

Dmitry, I’m happy to say that we’re planning to add concurrent actions in the pipelines in the next sprint.
We shall deliver them at the turn of 2017/2018.


#11

Hello @Octavia,

Thanks for the reply. Looking forward to see concurrent actions in Buddy.

I have only one question - would concurrent actions somehow affect pricing policy? For example, we would need to purchase some new plan(specifically interested in standalone Enterprise version) or pay some fixed amount, etc.

It’s really important for me to know because if pricing policy would be changed I need to explain my chief that original pricing estimate isn’t accurate.

Looking forward to your comments.

With best regards,
Dmitry


#12

Parallel actions use the same limits as concurrent releases. So, if we want 2 actions to be run at the same time, we should have 2 concurrent runs.

There’s one concurrent run on the free standalone version, so this option will not be available here. In the enterprise version, however, there are 4 concurrent runs set by default.


#13

Hello @Octavia,

thank you for the answer.

So if we would purchase standalone enterprise version (PREMIUM plan) we would be able to run 4 concurrent actions at the same time?

But would we have an ability to have more concurrent runs? With Cloud plans it’s pretty straight-forward about concurrent runs but as for Enterprise version I got a bit lost.
Would we be able to use as many concurrent runs as we need or we would be tight to 4 pre-defined?

With best regards,
Dmitry


#14

Dmitry, there are 4 concurrent runs set on default. However, if you contact us at:support@buddy.works, we’ll set as many concurrent runs for your license as you want.

Soon, we’ll add to the settings a possibility to change the limit.