Setting Up a Local Mirror for Composer Packages With Satis

Installing all your PHP libraries with Composer is a great way to save time. But larger projects automatically tested and run at each commit to your software version control (SVC) system will take a long time to install all the required packages from the Internet. You want to run your tests as soon as possible through your continuous integration (CI) system so that you have fast feedback and quick reactions on failure. In this tutorial we will set up a local mirror to proxy all your packages required in your project's composer.json file. This will make our CI work much faster, install the packages over the local network or even hosted on the same machine, and make sure we have the specific versions of the packages always available.


What Is Satis?

Satis is the name of the application we will use to mirror the various repositories for our project. It sits as a proxy between the Internet and your composer. Our solution will create a local mirror of a few packages and instruct our composer to use it instead of the sources found on the Internet.

Here is an image that says more than a thousand words.

Architecture

Our project will use composer as usual. It will be configured to use the local Satis server as the primary source. If a package is found there, it will be installed from there. If not, we will let composer use the default packagist.org to retrieve the package.


Getting Satis

Satis is available through composer, so installing it is very simple. In the attached source code archive, you will find Satis installed in the Sources/Satis folder. First we will install composer itself.

Then we will install Satis.


Configuring Satis

Satis is configured by a JSON file very similar to composer. You can use whatever name you want for your file and specify it for usage later. We will use "mirrored-packages.conf".

Let's analyze this configuration file.

  • name - represents a string that will be shown on the web interface of our mirror.
  • homepage - is the web address where our packages will be kept. This does not tell our web server to use that address and port, it is rather just information of a working configuration. We will set up the access to it on that addres and port later.
  • repositories - a list of repositories ordered by preference. In our example, the first repository is a Github fork of the monolog logging libraries. It has some modifications and we want to use that specific fork when installing monolog. The type of this repository is "vcs". The second repository is of type "composer". Its URL is the default packagist.org site.
  • require - lists the packages we want to mirror. It can represent a specific package with a specific version or branch, or any version for that matter. It uses the same syntax as your "require" or "require-dev" in your composer.json.
  • require-dependencies - is the final option in our example. It will tell Satis to mirror not only the packages we specified in the "require" section but also all their dependencies.

To quickly try out our settings we first need to tell Satis to create the mirrors. Run this command in the folder where you installed Satis.

While the process is taking place, you will see how Satis mirrors each found version of the required packages. Be patient it may take a while to build all those packages.

Satis requires that date.timezone to be specified in the php.ini file, so make sure it is and set to your local timezone. Otherwise an error will appear.

Then we can run a PHP server instance in our console pointing to the recently created repository. PHP 5.4 or newer is required.

And we can now browse our mirrored packages and even search for specific ones by pointing our web browser to http://localhost:4680:

MirrorWebpage

Let's Host It on Apache

If you have a running Apache at hand, creating a virtual host for Satis will be quite simple.

We just use a .conf file like this, put in Apache's conf.d folder, usually /etc/apache2/conf.d. It creates a virtual host on the 4680 port and points it to our folder. Of course you can use whatever port you want.


Updating Our Mirrors

Satis can not automatically update the mirrors unless we tell it. So the easiest way, on any UNIX like system, is to just add a cron job to your system. That would be very easy, and just a simple script to execute our update command.

The drawback of this solution is that it is static. We have to manually update the mirrored-packages.conf every time we add another package to our project's composer.json. If you are part of a team in a company with a big project and a continuous integration server, you can't rely on people remembering to add the packages on the server. They may not even have permissions to access the CI infrastructure.


Dynamically Updating Satis Configuration

It's time for a PHP TDD exercise. If you just want your code ready and running, check out the source code attached to this tutorial.

As usual we start with a degenerative test, just enough to make sure we have a working testing framework. You may notice that I have quite a strange looking require_once line, this is because I want to avoid having to reinstall PHPUnit and Mockery for each small project. So I have them in a vendor folder in my NetTuts' root. You should just install them with composer and drop the require_once line altogether.

That looks about right. All the fields except "require" are static. We need to generate only the packages. The repositories are pointing to our private git clones and to packagist as needed. Managing those is more of a sysadmin job than a software developer's.

Of course this fails with:

Fixing that is easy.

I just added an empty method with the required name, as private, to our test class. Cool, but now we have another error.

So, null does not match our string containing all that default configuration.

OK, that works. All tests are passing.

But we introduced a horrible duplication. All that static text in two places, written character by character in two different places. Let's fix it:

Ahhh! That's better.

Well. That also passes. But it also highlights some duplication and useless assignment.

We inlined the $expected variable. $actual could also be inlined, but I like it better this way. It keeps the focus on what is tested.

Now we have another problem. The next test I want to write would look like this:

But after writing the simple implementation, we will notice it requires json_decode() and json_encode(). And of course these functions reformat our string and matching strings will be difficult at best. We have to take a step back.

We changed our assertion method to compare JSON strings and we also recoded our $actual variable. ParseComposerConf() was also modified to use this method. You will see in a moment how it helps us. Our next test becomes more JSON specific.

And making this test pass, along with the rest of the tests, is quite easy, again.

We take the input JSON string, decode it, and if it contains a "require" field we use that in our Satis configuration file instead. But we may want to mirror all versions of a package, not just the last one. So maybe we want to modify our test to check that the version is "*" in Satis, regardless of what exact version is in composer.json.

That obviously fails with a cool message:

Now, we need to actually edit our JSON before re-encoding it.

To make the test pass we have to iterate over each element of the required packages array and set their version to '*'. See method toAllVersion() for more details. And to speed up this tutorial a little bit, we also extracted some private methods in the same step. This way, parseComoserConf() becomes very descriptive and easy to understand. We could also inline $config into the arguments of addNewRequires(), but for aesthetic reasons I left it on two lines.

But what about "require-dev" in composer.json?

That obviously fails. We can make it pass with just copy/pasting our if condition in addNewRequires():

Yep, that makes it pass, but those duplicated if statements are nasty looking. Let's deal with them.

We can be happy again, tests are green and we refactored our code. I think only one test is left to be written. What if we have both "require" and "require-dev" sections in composer.json?

That fails because the packages set by "require-dev" will overwrite those of "require" and we will have an error:

Just add a plus sign to merge the arrays, and we are done.

Tests are passing. Our logic is finished. All we left to do is to extract the methods into their own file and class. The final version of the tests and the SatisUpdater class can be found in the attached source code.

We can now modify our cron script to load our parser and run it on our composer.json. This will be specific to your projects' particular folders. Here is an example you can adapt to your system.


Making Your Project Use the Mirror

We talked about a lot of things in this article, but we did not mention how we will instruct our project to use the mirror instead of the Internet. You know, the default is packagist.org? Unless we do something like this:

That will make your mirror the first choise for composer. But adding just that into the composer.json of your project will not disable access to packagist.org. If a package can not be found on the local mirror, it will be downloaded from the Internet. If you wish to block this feature, you may also want to add the following line to the repositories section above:


Final Thoughts

That's it. Local mirror, with automatic adapting and updating packages. Your colleagues will never have to wait a long time while they or the CI server installs all the requirements of your projects. Have fun.

Tags:

Comments

Related Articles