items tagged with ant

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage. Martin Fowler about continuous integration

Things get clearer in my mind...I will use a set of Open Sources software to reach this ambitious goal.

Brainstorming

Maintain a Single Source Repository.

Either Subversion running locally on windows/Linux, but I will stay with www.joomlaCode.org at the beginning

Automate the Build

• I am currently evaluation Maven for PHP but already have ANT script to build components and plugin for Joomla 1.5 (they are not generic enough at the moment)

Make Your Build Self-Testing

• PHP UNIT tests if available with be run at this stage using PHP command line.
• Selenium to automate components testing across many platforms, as it runs in many browsers and operating systems, and can be controlled by many programming languages and testing frameworks.
• Some standard utilities to perform standard Joomla! processes: login, logout for Apache ANT or Selenium
• I will develop either a ANT plugin or a set of Selenium test cases for deploying and removing
  • component: install, publish, remove
  • plugin: install, publish, remove
• All these utilities will be available under GPL v3 and as such free to reuse, and improve.

Everyone Commits Every Day

• This is reserved to developer having a www.joomlaCode.org account and part of the development team.

Every Commit Should Build the Mainline on an Integration Machine

I will use TeamCity continuous build server as it is free for personal use (limited to 20 concurrent build), run on Windows and Linux but require a java VM to run (may not be wanted in a PHP environment). Anyway with ANT, it will be possible to use another build server like Cruise Control or PHP Under Control.

Keep the Build Fast

That is an objective :-)

Test in a Clone of the Production Environment

Joomla! Build farm

• I can imagine a set of Joomla instances, ideally 5 of each version, aka Joomla! 1.5.3 to Joomla! 1.5.8 and Joomla! 1.0.10 to 1.0.15
• Joomla instances will be recreated at build time (files and databases), that mean Joomla! will get newly installed and removed in case of successful build
• All Joomla! instances will be running with XAMPP, ideally on port not available to the outside world for security reasons

Make it Easy for Anyone to Get the Latest Executable

Successful build (Artifacts) are only available if build is successful. Team City provide this with less effort (configuration)

Everyone can see what's happening

• A guest account will be available or a free public area with limited access to see the result of builds.
• RSS feeds, emails and Instant messaging (Jabber) out of the box for end users or developers!

Automate Deployment

That will be, auto publish to some demo site in a configurable way. At the moment, at http://demo.waltercedric.com and http://demo2.waltercedric.com for me :-)

Final words

• I will provide a ready to use package for Windows and Linux and all scripts, so anybody will be able to run it also on your own.
• Critical part will be documented in my WIKI at http://wiki.waltercedric.com direct link HERE

It seem that nobody is providing such a package as I am after only one day at the top of search results in Google "continuous build joomla"

Automatic installation of Joomla! runtime environments

Main ideas

Build is scalable

Distributed build management optimize hardware resources utilization by parallelizing product builds within the build agents grid. With build chains support, it is even possible to break down a single build procedure into several parts to run them on different build agents — both in sequence and in parallel — using the same set of sources in all of them.

I want to be able to test my components against many versions of Joomla!

All versions of Joomla! are in subversion as zip files in an own SVN repository

For example:

• ${JOOMLA15_VCS_ROOT}  is svn:\\localhost\joomla1.5\trunk
• ${JOOMLA10_VCS_ROOT}  is svn:\\localhost\joomla1.0\trunk

These repository ${JOOMLAxx_VCS_ROOT} are connected to all build as supplementary VCS root in TeamCity and thus content get checked out as part of the build in the build temporary directory of one agent. ($AGENT_BUILD_DIR)

joomla1.5\trunk
                            Joomla_1.5.4-Stable-Full_Package.zip
                            Joomla_1.5.5-Stable-Full_Package.zip
                            Joomla_1.5.6-Stable-Full_Package.zip
                            Joomla_1.5.7-Stable-Full_Package.zip
                            Joomla_1.5.8-Stable-Full_Package.zip

So after the checkout, the file system will look like

($AGENT_BUILD_DIR)\                     

                                 Joomla_1.5.4-Stable-Full_Package.zip
                            Joomla_1.5.5-Stable-Full_Package.zip
                            Joomla_1.5.6-Stable-Full_Package.zip
                            Joomla_1.5.7-Stable-Full_Package.zip
                            Joomla_1.5.8-Stable-Full_Package.zip

If you don't want to provide support a a specific version of Joomla! just remove it from the trunk! or add new ones on purpose. That's easy.

Ant tasks/Maven MOJO

• Are responsible for unpacking all these zip files to the build temporary agent directory. ($AGENT_BUILD_DIR).
• Filenames are found with a configurable regular expression,
• All settings will be committed in \joomla1.5\trunk\build.deploy.properties

Another ant script/task will for each zip,

• Start a Selenium test cases that will create a virtual user that use the regular Joomla! installer and drive installation till the end.
• All settings which have to be Joomla! and build independent will be randomly generated, preferably UUID for password and database name for example.
• Login and Admin password may be the same (admin:admin) at the beginning but can also be generated and written to a file on disk in ($BUILD_DIR)/{joomlarootversion}/build.install.properties.
• Directory installation ($BUILD_DIR)/{joomlarootversion}/installation will be renamed to ($BUILD_DIR)/{joomlarootversion}/installation.old or simply deleted
• Selenium/PHP Unit test that are committed in \joomla1.5\trunk\Installation.Checks will perform basic checks (login, logout, navigate) to ensure that installation of Joomla! has been successful.
  If everything succeed, we will have a set of Joomla! versions ready for our components regression testing.

Remarks:

• No build temporary directory. ($AGENT_BUILD_DIR) will be deleted by Ant or Maven but by the build server itself. This will let developer look at the issues on file system and in database.
• New scripts may be developed to extract from the build server or Joomla! farm easily the non running Joomla! instance files + database) so developers can install the broken setup locally.

Automatic deployment of Joomla! components

Your component is typically shared and many developer committed regularly in a different VCS root... For SecurityImages 5.x.y, subversion root may be  svn:\\localhost\securityimages5\trunk

This VCS root is also attached to the build and get check out at build time by TeamCity.

Packaging

if a build.xml is present in {VCS_ROOT}\build.xml then it is executed prior to any further operations, purpose of build.xml is to produce a component binary distribution (zip or tar.gz) that can be then installed to ALL Joomla install in the agent root directory.

Deployment

if a deploy.xml is present in {VCS_ROOT}\deploy.xml then it is executed, purpose of deploy.xml is to deploy one or many component binary distribution (zip or tar.gz) to ALL Joomla install in the agent root directory.

Why one or many component?

I want to be able to track also component dependencies issues.

Lets say that SecurityImages does not play well with VirtueMart, I may want to test also that combination across Joomla! instances, that's why VirtueMart may have to be deployed with SecurityImages or not.

prerequisites:

• Running SVN server, see HERE for installing it on windows
• Installed JVM, latest JDK 1.6u10
• Running TeamCity server
• Running XAMPP with HTTP root directory at TeamCity agent root directory.
• Apache ANT with additional library for more control (if, case, for loop)

This articles will be available in my WIKI soon http://wiki.waltercedric.com/index.php?title=ContinuousBuildforJoomla so any reader or developer can participate to the discussion, next step is to implement the above and that will e documented as well :-)

I will commit soon a first draft (alpha) of what is expected to bring continuous build to any Joomla! component (or event to Joomla! core itself ;-))

You are free to join the project, all documentation effort stay at the moment in my WIKI