teamcity

TeamCity

TeamCity is a Java-based build management and continuous integration server from JetBrains. [http://en.wikipedia.org/wiki/TeamCity]

  • Apache Maven manifest and enabling agile testing

    apache_maven

    In this small post i will present you how to

    1. Track and identify across your Apache Maven multi project builds all components, their versions, and class path if needed
    2. Get valuable information from your testing team, and add it to the GUI of your web applications

    To solve the problem number 1, we will use Manifest files

    On the Java platform, a manifest file is a specific file contained within a JAR archive. It is used to define extension and package related data. It is a metadata file that contains name-value pairs organized in different sections. If a JAR file is intended to be used as an executable file, the manifest file specifies the main class of the application. The manifest file is named MANIFEST.MF. [Wikipedia]

    If you do nothing special in Maven., you will see across your jar files the following in all META-INF/MANIFEST.MF

    Manifest-Version: 1.0
    Archiver-Version: Plexus Archiver
    Created-By: Apache Maven

     

    The solution I propose here will write all these META-INF/MANIFEST.MF the following  content:

    Manifest-Version: 1.0
    Archiver-Version: Plexus Archiver
    Created-By: Apache Maven
    Built-By: agent1
    Build-Jdk: 1.6.0_20
    Specification-Title: Unnamed - com.waltercedric:web:war:1.0.0-SNAPSHOT
    Specification-Version: 1.0.0-SNAPSHOT
    Specification-Vendor: waltercedric
    Implementation-Title: Unnamed - com.waltercedric:web:war:1.0.0-SNAPSHOT
    Implementation-Version: 1.0.0-SNAPSHOT
    Implementation-Vendor-Id: com.waltercedric
    Implementation-Vendor: waltercedric
    buildDate: 2010.09.22-14:12
    svnrevision: 18625
    Class-Path: spring-ws-core-1.5.6.jar spring-aop-2.5.6.jar spring-
     web-2.5.6.jar spring-webmvc-2.5.6.jar spring-context-support-2.5.6.jar
    .. .. ..

     

  • Configuring TeamCity, Maven for PHP for Joomla continuous build

    apache_maven

    Doxygen phpDocumentator phpunit-logo teamcity512 maven4php

    Maven for PHP uses the power of Maven for building, reporting on and creating documentations of PHP projects. It adapts the Maven build lifecycle to the PHP world while fully supporting PHP 4 and PHP 5. PHP for Maven uses PHPUnitfor unit testing and doxygenfor creating the api documentation.
    Use a PHP library project to create a library that can be used by other PHP libraries or PHP web projects. Use a PHP web project to create a standalone web project.

    So I quickly describe what I did install on my root server (OpenSuse 11.X)

    My Objectives: being able to build all my Joomla! component using best agile development practices

    “Specific tools and techniques such as continuous integration, automated or xUnit test, pair programming, test driven development, design patterns, domain-driven design, code refactoring and other techniques are often used to improve quality and enhance project agility.”

    Maven

    While not needed as TeamCity has an integrated Maven engine, I would like to use an external MAVEN version, in order to have the latest version and living dangerously on the edge!

    So I download

    # wgethttp://apache.mirror.testserver.li/maven/binaries/apache-maven-2.1.0-bin.tar.gz

    And unpack

    # tar xvf apache-maven-2.1.0-bin.tar.gz

    Since I would like to avoid having a version number in my configuration build path, I create a symlink

    # ln -s apache-maven-2.1.0 maven

    I just now tell TeamCity to use my own Maven version, by specifying the Maven Home Path

    teamcity.maven4php 

    phpDocumentor

    phpDocumentor is an open source documentation generator written in PHP. It automatically parses PHP source code and produces readable API and source code documentation in a variety of formats. phpDocumentor generates documentation based on PHPDoc-formatted comments and the structure of the source code itself. It supports documentation of both object-oriented and procedural code. phpDocumentor can create documentation in HTML, PDF, CHM or Docbook formats.

    Can be installed using PEAR, simply run

    # pear upgrade PhpDocumentor
    downloading PhpDocumentor-1.4.2.tgz...
    Starting to download PhpDocumentor-1.4.2.tgz (2,421,028 bytes)
    ..............................................................................done: 2,421,028 bytes
    upgrade ok: channel://pear.php.net/PhpDocumentor-1.4.2

    PHPUnit

    PHPUnit is a unit testing framework for the PHP programming language. Created by Sebastian Bergmann, PHPUnit is one of the xUnit family of frameworks that originated with Kent Beck's SUnit.

    Can be installed using PEAR, simply run

    # pear upgrade PHPunit

    DOxygen

    Doxygen is a documentation generator for C++, C, Java, Objective-C, Python, IDL (Corba and Microsoft flavors), Fortran, VHDL, PHP, C#, and to some extent D. It runs on most Unix-like systems, including Mac OS X, as well as on Windows. The first version of Doxygen borrowed some code of an old version of DOC++; later, the Doxygen code was rewritten by Dimitri van Heesch. from www.doxygen.org

    # zypper se doxy
    Lese installierte Pakete...

    S | Name       | Zusammenfassung                                    | Typ 
    --+------------+----------------------------------------------------+------
      | doxygen    | Automated C, C++, and Java Documentation Generator | Paket
      | doxywizard | Graphical User Interface for Doxygen               | Paket

    # zypper in doxygen

    Herunterladen von Paket doxygen-1.5.5-17.1.x86_64 (1/1), 2,3 M (6,2 M installiert)
    Lade herunter: doxygen-1.5.5-17.1.x86_64.rpm [fertig] 
    Installiere: doxygen-1.5.5-17.1 [fertig]

    Artifactory

    Prepare Artifactory by adding new Maven for PHP repositories

    At http://maven.waltercedric.com/artifactory/webapp/repositoryconfig.html

    As Admin user (you cant go to that links without being an admin!), add 2 new repositories

    teamcity.maven4php.artifactory

    In Maven settings.xml

    In order to use Artifactory at his best (proxy and caching of remote repositories), I have a Maven settings.xml that contains ONLY

    # vi /home/teamcity/.m2/settings.xml

    <activeProfiles>
      <activeProfile>cedric-profile</activeProfile>
    </activeProfiles>

    <profiles>
       <profile>
            <id>cedric-profile</id>
            <activation>
            <activeByDefault>true</activeByDefault>
            </activation>
    <repositories>
        <repository>
            <id>central</id>
            <url>http://maven.waltercedric.com/artifactory/repo/</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
        <repository>
            <id>snapshots</id>
            <url>http://maven.waltercedric.com/artifactory/repo/</url>
            <releases>
                <enabled>false</enabled>
            </releases>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <id>central</id>
            <url>http://maven.waltercedric.com/artifactory/plugins-releases/</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </pluginRepository>
        <pluginRepository>
            <id>snapshots</id>
            <url>http://maven.waltercedric.com/artifactory/plugins-snapshots/</url>
            <releases>
                <enabled>false</enabled>
            </releases>
        </pluginRepository>
    </pluginRepositories>
        </profile>
      </profiles>
    </settings>

    Attention: Artifactory WIKI tell to use http://maven.waltercedric.com/artifactory/plugins-releases and not http://maven.waltercedric.com/artifactory/plugins-releases/ on my HOST with mod_proxy it made an error 404 If I do not add a slash at the end. Try with your host before, you will gain a lot of time by checking if the URL is valid!

    Note that http://url:port/artifactory/repo/ is a virtual repositories that proxy all external repositories

     

    Eclipse

    Use SolarJoomla (hopefully to be distributed this week) to have a running Eclipse, PDT, Maven 4 PHP environment, Mylyn, TeamCity in no time

    Lets build!

    First I create a new Maven Project with Archetype “Maven for PHP” “PHP5 libraries”

    In TeamCity I did create a new project “Joomla 15 components plugins and modules” and a new Build “MyGuestbook”.

    teamcity.maven4php.myguestbook

    The first build failed with

    [INFO] PHP Warning: require_once(PHPUnit/TextUI/TestRunner.php): failed to open stream: Operation not permitted

    This is because of my PHP security restrictions, I only allow file to be opened from /home/teamcity/

    So I just add

    /home/teamcity/TeamCity/buildAgent/ to my open_basedirin my php.ini

    ; open_basedir, if set, limits all file operations to the defined directory
    ; and below.  This directive makes most sense if used in a per-directory
    ; or per-virtualhost web server configuration file. This directive is
    ; *NOT* affected by whether Safe Mode is turned On or Off.
    open_basedir = /srv/www/vhosts:/tmp:/home/teamcity/TeamCity/buildAgent/

    To be continued

    So long an empty PHP project is building successfully, tomorrow I will try to make a REAL Joomla! component build there!

    As soon as It works, and all my Joomla! components are running in TeamCity, I will try to achieve the same goal in Bamboo, why? because it is simply . . .fun!

    Links

  • Continuous Build for Joomla

    Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage. Martin Fowler about continuous integration

    Things get clearer in my mind...I will use a set of Open Sources software to reach this ambitious goal.

    Brainstorming

    Maintain a Single Source Repository.

    Either Subversion running locally on windows/Linux, but I will stay with www.joomlaCode.org at the beginning

    Automate the Build

    • I am currently evaluation Maven for PHP but already have ANT script to build components and plugin for Joomla 1.5 (they are not generic enough at the moment)

    Make Your Build Self-Testing

    • PHP UNIT tests if available with be run at this stage using PHP command line.
    • Seleniumto automate components testing across many platforms, as it runs in many browsers and operating systems, and can be controlled by many programming languages and testing frameworks.
    • Some standard utilities to perform standard Joomla! processes: login, logout for Apache ANT or Selenium
    • I will develop either a ANT plugin or a set of Selenium test cases for deploying and removing
      • component: install, publish, remove
      • plugin: install, publish, remove
    • All these utilities will be available under GPL v3 and as such free to reuse, and improve.

    Everyone Commits Every Day

    • This is reserved to developer having a www.joomlaCode.org account and part of the development team.

    Every Commit Should Build the Mainline on an Integration Machine

    I will use TeamCity continuous build server as it is free for personal use (limited to 20 concurrent build), run on Windows and Linux but require a java VM to run (may not be wanted in a PHP environment). Anyway with ANT, it will be possible to use another build server like Cruise Control or PHP Under Control.

    Keep the Build Fast

    That is an objective :-)

    Test in a Clone of the Production Environment

    Joomla! Build farm

    • I can imagine a set of Joomla instances, ideally 5 of each version, aka Joomla! 1.5.3 to Joomla! 1.5.8 and Joomla! 1.0.10 to 1.0.15
    • Joomla instances will be recreated at build time (files and databases), that mean Joomla! will get newly installed and removed in case of successful build
    • All Joomla! instances will be running with XAMPP, ideally on port not available to the outside world for security reasons

    Make it Easy for Anyone to Get the Latest Executable

    Successful build (Artifacts) are only available if build is successful. Team City provide this with less effort (configuration)

    Everyone can see what's happening

    • A guest account will be available or a free public area with limited access to see the result of builds.
    • RSS feeds, emails and Instant messaging (Jabber) out of the box for end users or developers!

    Automate Deployment

    That will be, auto publish to some demo site in a configurable way. At the moment, at http://demo.waltercedric.com and http://demo2.waltercedric.com for me :-)

    Final words

    • I will provide a ready to use package for Windows and Linux and all scripts, so anybody will be able to run it also on your own.
    • Critical part will be documented in my WIKI at http://wiki.waltercedric.com direct link HERE

    It seem that nobody is providing such a package as I am after only one day at the top of search results in Google "continuous build joomla"

  • Continuous build for Joomla! part1/x

    Automatic installation of Joomla! runtime environments

    Main ideas

    Build is scalable

    Distributed build management optimize hardware resources utilization by parallelizing product builds within the build agents grid. With build chains support, it is even possible to break down a single build procedure into several parts to run them on different build agents — both in sequence and in parallel — using the same set of sources in all of them.

    I want to be able to test my components against many versions of Joomla!

    All versions of Joomla! are in subversion as zip files in an own SVN repository

    For example:

    • ${JOOMLA15_VCS_ROOT}  is svn:\\localhost\joomla1.5\trunk
    • ${JOOMLA10_VCS_ROOT}  is svn:\\localhost\joomla1.0\trunk

    These repository ${JOOMLAxx_VCS_ROOT} are connected to all build as supplementary VCS root in TeamCity and thus content get checked out as part of the build in the build temporary directory of one agent. ($AGENT_BUILD_DIR)

    joomla1.5\trunk
                                Joomla_1.5.4-Stable-Full_Package.zip
                                Joomla_1.5.5-Stable-Full_Package.zip
                                Joomla_1.5.6-Stable-Full_Package.zip
                                Joomla_1.5.7-Stable-Full_Package.zip
                                Joomla_1.5.8-Stable-Full_Package.zip

    So after the checkout, the file system will look like

    ($AGENT_BUILD_DIR)\                     

                                     Joomla_1.5.4-Stable-Full_Package.zip
                                Joomla_1.5.5-Stable-Full_Package.zip
                                Joomla_1.5.6-Stable-Full_Package.zip
                                Joomla_1.5.7-Stable-Full_Package.zip
                                Joomla_1.5.8-Stable-Full_Package.zip

    If you don't want to provide support a a specific version of Joomla! just remove it from the trunk! or add new ones on purpose. That's easy.

    Ant tasks/Maven MOJO

    • Are responsible for unpacking all these zip files to the build temporary agent directory. ($AGENT_BUILD_DIR).
    • Filenames are found with a configurable regular expression,
    • All settings will be committed in \joomla1.5\trunk\build.deploy.properties

    Another ant script/task will for each zip,

    • Start a Selenium test cases that will create a virtual user that use the regular Joomla! installer and drive installation till the end.
    • All settings which have to be Joomla! and build independent will be randomly generated, preferably UUID for password and database name for example.
    • Login and Admin password may be the same (admin:admin) at the beginning but can also be generated and written to a file on disk in ($BUILD_DIR)/{joomlarootversion}/build.install.properties.
    • Directory installation ($BUILD_DIR)/{joomlarootversion}/installation will be renamed to ($BUILD_DIR)/{joomlarootversion}/installation.old or simply deleted
    • Selenium/PHP Unit test that are committed in \joomla1.5\trunk\Installation.Checks will perform basic checks (login, logout, navigate) to ensure that installation of Joomla! has been successful.
      If everything succeed, we will have a set of Joomla! versions ready for our components regression testing.

    Remarks:

    • No build temporary directory. ($AGENT_BUILD_DIR) will be deleted by Ant or Maven but by the build server itself. This will let developer look at the issues on file system and in database.
    • New scripts may be developed to extract from the build server or Joomla! farm easily the non running Joomla! instance files + database) so developers can install the broken setup locally.

    Automatic deployment of Joomla! components

    Your component is typically shared and many developer committed regularly in a different VCS root... For SecurityImages 5.x.y, subversion root may be  svn:\\localhost\securityimages5\trunk

    This VCS root is also attached to the build and get check out at build time by TeamCity.

    Packaging

    if a build.xml is present in {VCS_ROOT}\build.xml then it is executed prior to any further operations, purpose of build.xml is to produce a component binary distribution (zip or tar.gz) that can be then installed to ALL Joomla install in the agent root directory.

    Deployment

    if a deploy.xml is present in {VCS_ROOT}\deploy.xml then it is executed, purpose of deploy.xml is to deploy one or many component binary distribution (zip or tar.gz) to ALL Joomla install in the agent root directory.

    Why one or many component?

    I want to be able to track also component dependencies issues.

    Lets say that SecurityImages does not play well with VirtueMart, I may want to test also that combination across Joomla! instances, that's why VirtueMart may have to be deployed with SecurityImages or not.

    prerequisites:

    • Running SVN server, see HERE for installing it on windows
    • Installed JVM, latest JDK 1.6u10
    • Running TeamCity server
    • Running XAMPP with HTTP root directory at TeamCity agent root directory.
    • Apache ANT with additional library for more control (if, case, for loop)

    This articles will be available in my WIKI soon http://wiki.waltercedric.com/index.php?title=ContinuousBuildforJoomla so any reader or developer can participate to the discussion, next step is to implement the above and that will e documented as well :-)

  • ContinuousBuild4Joomla project submitted to JoomlaCode.org

    I will commit soon a first draft (alpha) of what is expected to bring continuous build to any Joomla! component (or event to Joomla! core itself ;-))

    You are free to join the project, all documentation effort stay at the moment in my WIKI

     

  • Development shift in the way I deliver my latest Joomla! extensions

    agile.development

     

    Starting from now on, I will deliver all my latest (unstable) extensions versions through  my continuous build server. All request or bugs discovery that are requested in my forums and solved will lead to a new build that you will be able to download a lot faster than before.

    Thanks to Maven for PHP, I can now commit, 60 seconds later, unit test run and  the result is a direct download for my extensions snapshots.

     

     

     

     

     

     

    Here is an example with the module mod_related_thumb_items

    Head to http://teamcity.waltercedric.com/teamcity/guestLogin.html?guest=1

    Locate the module or component you are interested in:

    HowTODownloadLatest

    Click on the latest build, must be  Success

    HowTODownloadLatest.01  

    If this build is a direct answer to a support request in my forums, or solve an issue, You should be able to see in changes the commit description, and even which file have been changed after and before the commit.

    http://teamcity.waltercedric.com/teamcity/viewLog.html?buildId=217&buildTypeId=bt3&tab=buildChangesDiv

    HowTODownloadLatest.05

    But Hey! you want to download this latest build now, go to artifact

    HowTODownloadLatest.02

    Staying  on the edge by using RSS

    You can monitor any build by using the RSS icon in your browser toolbar, or example with this module, it would be

    http://teamcity.waltercedric.com/guestAuth/feed.html?buildTypeId=bt20&itemsType=builds&userKey=guest

    This way of downloading the latest extensions do not replace the page http://www.waltercedric.com/joomla-releases-mainmenu-269.html where there is there only stable versions.

    The next step is to make the maven phase “site” work (I have issue with phpdocumentor not found), this will create automatically a internet site in one of my sub-domains ad hide this complexity.

  • Display the short GIT version hash as build identifier in TeamCity

    TeamCity is a Continuous integration and build management server from JetBrains.

    As the first build step, create a custom script that reads %build.number%, and uses service messages to append the short GIT hash programmatically.

    image

    Here’s an example using a bash script pasted right into the TeamCity GUI (Runner type: Command Line, Run: Custom Script):

    GIT_HASH=%build.vcs.number%
    GIT_HASH_SHORT=${GIT_HASH:0:7}
    echo "##teamcity[buildNumber '#${GIT_HASH_SHORT}']"
     
  • git-branch-renamer-maven-plugin

    When working with many feature/release/bugix/hotfix branches, it is a bad idea to start changing the pom version as this will create merge conflicts using pull request. this plugin allow you to keep in ALL branches the same pom version for all your projects, for example MASTER-SNAPSHOT the version will be derived from branch name automagically :-)

    You may want to read more first these 2 short articles

    git-branch-renamer-maven-plugin allow you to keep in ALL branches the same pom version for all your projects: for example MASTER-SNAPSHOT and never change it again.

    the project version will be derived from branch name automatically when running in your continuous integration server.

    branch name feature/xxxx

    • <version>xxxx-SNAPSHOT</version> (default)
    • <version>xxxx</version> (release = true)
    • <version>0-xxxx-SNAPSHOT</version> (forceNumericalVersion = true)
    • <version>feature-xxxx-SNAPSHOT</version> (filterOutBranchQualifier = false)

    The project is hosted at Github https://github.com/cedricwalter/git-branch-renamer-maven-plugin 

  • Installing TeamCity standalone on OpenSuse 11.x/ Plesk 9

    teamcity512

    After fighting many days.. with their container versions (many webapps in the same tomcat with mod_proxy, mod_proxy_ajp), I did install JetBrains TeamCity, Atlassian Jira and Bamboo as standalone on my server.

    TeamCity is a Java-based build management and continuous integration server from JetBrains, creators of IntelliJ IDEA and ReSharper.

     

    What will happen next?

     

    Better run these webapps with their own user for more security…

    # groupadd teamcity
    # useradd -g teamcity -c 'Build user for teamcity' -m teamcity

    Get the latest versions of the software
    # wgethttp://download.jetbrains.com/teamcity/TeamCity-4.0.2.tar.gz

    I will install everything in the default home of that user teamcity, so I move there
    # cd /home/teamcity

    Root magic in action
    # su – teamcity

    Unpack archive
    # tar xvf
    TeamCity-4.0.2.tar.gz

    Define port

    Teamcity is shipped with a Tomcat instance, I slightly modify it as OpenSuse is already using a Tomcat that may be running (HTTP 8080- Admin 8005 – AJP 8009)

    # vi /home/teamcity/TeamCity/conf/server.xml

    • HTTP 8111 (wont be accessible by firewall rule)

      <Connector port="8111" protocol="HTTP/1.1"
                    connectionTimeout="20000"
                    redirectPort="8443"
                    enableLookup="false"        
                    useBodyEncodingForURI="true"
             />

    • Admin 8006  (default is 8005)

      <Server port="8006" shutdown="SHUTDOWN">

    • AJP 8010  (default is 8009)  <Connector port="8010" protocol="AJP/1.3" redirectPort="8443" />

    Worker.properties

    Now I tell apache that there is a new AJP waiting on port 8010

    # vi /etc/apache/worker.properties

    worker.list=ajp13, teamcity
    worker.ajp13.port=8009
    worker.ajp13.host=localhost
    worker.ajp13.type=ajp13

    worker.teamcity.port=8010
    worker.teamcity.host=localhost
    worker.teamcity.type=ajp13

    Plesk

    Add  new subdomains, I have defined

    Virtual host

    create 2 vhost.conf files

    • /srv/www/vhosts/waltercedric.com/subdomains/teamcity/conf/vhost.conf
    • /srv/www/vhosts/waltercedric.com/subdomains/continuousbuildserver/conf/vhost.conf

    # vi /srv/www/vhosts/waltercedric.com/subdomains/continuousbuildserver/conf/vhost.conf

    ServerName continuousbuildserver.waltercedric.com

    ProxyPass /  ajp://continuousbuildserver.waltercedric.com:8010/
    <Proxy *>
       Order Allow,Deny
       Allow from all
    </Proxy>

    <Directory />
      Options FollowSymLinks
      AllowOverride None
    </Directory>

    # vi /srv/www/vhosts/waltercedric.com/subdomains/teamcity/conf/vhost.conf

    ServerName teamcity.waltercedric.com

    ProxyPass /  ajp://teamcity.waltercedric.com:8010/
    <Proxy *>
       Order Allow,Deny
       Allow from all
    </Proxy>

    <Directory />
      Options FollowSymLinks
      AllowOverride None
    </Directory>

    To reread and include overridden vhost.conf, and tell plesk that you have create new vhost.conf, run at least once 
    # /usr/local/psa/admin/sbin/websrvmng

    Restart now apache

    # rcapache2 restart

    Start TeamCity

    # /home/teamcity/Teamcity/bin/runAll.sh start

    Do some basic checks to see that everything run or look at logs tail –f /home/teamcity/Teamcity/logs/catalina.out

    # netstat –an | grep 8010

    # netstat –an | grep 8011

     

    You’re done with TeamCity!

    Visit now either

  • http://teamcity.waltercedric.com Or
  • http://continuousbuildserver.waltercedric.com

  • Maven dependencies graph plugin in TeamCity and Eclipse

    apache_maven

    How to add dependencies graph to multi module projects. With this Maven plugin, you’ll be able to visualize Maven modules interdependencies and dependencies in any scope (compile, text, provided, system, runtime)

    depgraph:depgraph Can be used to draw a dependency graph from the project, the mojo is executed in. It traverses all dependencies and creates a graph using Graphviz. It draws a dependency graph just for your project. For a simple POM with no sub modules, it draws a graph of all dependencies (including transitive ones) below it. For a POM with sub modules, goes into each leaf POM and generates a separate graph for it.

     

    Here is an example of output on the plugin itself

    depgraph

     

    Install in all TeamCity Agent Graphwiz

    Graphviz is an open source graph visualization software. It has several main graph layout programs. See the gallery for some sample layouts. It also has web and interactive graphical interfaces, and auxiliary tools, libraries, and language bindings.

    Chance is that you are using Linux, so install is very easy and just a few click away for OpenSuse

    # zypper in graphwiz   

    or for Debian

    # apt-get install graphwiz

    In windows use the binary installer and put the graphwiz/bin in your PATH environment variable!

    Configure your POM

    Ideally put this in your parent pom inside the <build> </build> tag

    <plugin>
            <groupId>ch.elca.el4j.maven.plugins</groupId>
            <artifactId>maven-depgraph-plugin</artifactId>
            <version>1.7</version>
    </plugin>

    More configuration settings can be found HERE, now add either a new plugin repository location in your pom.xml (see below) or better in your artifactory proxy

      <pluginRepository>
        <id>elca-services</id>
        <url>http://el4.elca-services.ch/el4j/maven2repository</url>
        <releases>
         <enabled>true</enabled>
        </releases>
      </pluginRepository>

     

    Configure Teamcity build

    Add in the Maven runner of every TeamCity Build

    addDependenciesGraphGoalsInBuild

     

    Maven goals

    • depgraph:depgraph  Can be used to draw a dependency graph from the project, the mojo is executed in. It traverses all dependencies and creates a graph using Graphviz. It draws a dependency graph just for your project. For a simple POM with no submodules, it draws a graph of all dependencies (including transitive ones) below it. For a POM with submodules, goes into each leaf POM and generates a separate graph for it.
    • depgraph:fullgraph  Can be used to draw a dependency graph from the project, the mojo is executed in. It traverses all dependencies and creates a graph using Graphviz. It draws a graph for all the modules as they are interconnected. Same as depgraph for a simple POM, but for a POM with submodules, generates a combined dependency graph incorporating all modules.

    You may also want to let developer look at modules dependencies graph in TeamCity, so you may want to add to artifact path **/site/images/*.png => dependenciesGraph

    Artifacts are files produced by a build. After finishing a build, TeamCity searches for artifacts in the build's checkout directory according to the specified artifact patterns. Matching files are then uploaded to the server, where they become available for download. More ..

    artifactPath

     

    Configure Eclipse

    Install Graphviz and don’t forget to have it in PATH.

    You can share an eclipse Maven launcher in your parent project, right click on your pom.xml and select run as Maven configuration, specify either depgraph:fullgraph  or depgraph:depgraph   as goals

  • Module Jetbrains Teamcity for Joomla!

    Teamcityis a distributed build management and Continuous Integration system that allows your team to run and monitor your software building process while improving team communication, productivity and the integration of changes.

    This extension displays the status of your TeamCitycontinuous integration builds. TeamCityREST API is extremely rich, and I did made some choice concerning the ouput. Feel free to provide and send me ideas how I can improve the layout or by adding new functionalities!

    teamcity.module.for.joomla.frontend.01 teamcity.module.for.joomla.frontend

    The right picture also show how build status is represented with different colors (can be changed in CSS file):

    • green: last build was successful
    • red: last build failed
    • orange: last build has warning
    • gray: never build yet!
    • yellow: notice

    Features

    • Native Joomla! 1.5 module,
    • CSS driven output (media/mod_teamcity/css/teamcity.css)
    • Code produced is XHTML 1.0 Strict,
    • Full module caching for better performances
    • Tested in the following browser Firefox, Opera, Chrome, IE 8
    • Approved at Joomla Extension JED, NOT YET BUT PENDING
    • GPL v3.0,
    • Non commercial.

    Limitations

    1. Only tested with Teamcity 5

    Download mod_teamcity_1.0.0.zip

    Wiki

  • My 2 first Joomla! component in continuous build

    teamcity.joomla

    Status/Done

    • Joomla is also a simple dependency in scope compile (productive code and tests need it)
      <?xml version="1.0" encoding="UTF-8"?>
      <project>
        <modelVersion>4.0.0</modelVersion>
        <groupId>org.joomla</groupId>
        <artifactId>joomla</artifactId>
        <version>1.5.10</version>
        <description></description>
      </project>

      Deployed and viewable at http://maven.waltercedric.com/artifactory/webapp/browserepo.html
      joomla.in.artifactory

      So binding your code to a new version or old version is as simple as changing in your component pom.xml

      <dependency>
          <groupId>org.joomla</groupId>
          <artifactId>joomla</artifactId>
          <version>1.5.10</version> 
      </dependency>


     

    Delayed Commit (also known as private build). It allows you to run the full build with tests on the server as if you checked in all your changes, but without actually checking in your changes until the build is successful,so you will know if you're about to break the build before you actually break it.

    Open

    • Deploy generated artifacts to a Joomla running stage (runtime farms of Joomla!) in order to run integration testswith Selenium
    • Add selenium test cases for testing the GUI, also running “integration tests
    • Packaging is jar, better would be zip or tar.gz
    • Use Phing (http://docs.joomla.org/Setting_up_your_workstation_for_extension_development) for developer environment, and maybe also deploy to farms
    • Deliver SolarJoomla as soon as PHPDocumentator, PHPUnit, DOxygen are running in XAMPP
    • Publish generated site artifacts' to new sub domains
    • Add reporting plugin.
    • DOCUMENT everything in myWIKI at the same time

    Future

    Must work also in Atlassian Bamboo, Apache Hudson … I need your help for that, that wont be a huge task.

  • Status of the ContinuousBuild4Joomla framework

    I put some effort the last few days in this new framework.

    Done:

    • I did document some part of it at http://wiki.waltercedric.com/index.php?title=ContinuousBuildforJoomla
    • TeamCity is installed/configured/documented (windows only)
    • XAMPP is installed/configured/documented (windows only)
    • I also at the same time configured a Puppy Linux VMWare image that will permit to anybody to have a running environment in no time.
    • I am able to unpack all Joomla! versions that are committed in any repository (CVS, SVN, Clearcase)
    • They can be unpacked anywhere on file system (config in setEnv.xml), ideally they are unpacked in the root htdocs of XAMPP
    • Code is committed at Joomla forge in SVN http://joomlacode.org/gf/project/continbuild4j/

    Issues

    Selenium test suite do not accept a baseurl (or only a part of it) so I have a full path like /Joomla1.5.7/installation/index.php in a selenium test case instead of /installation/index.php in test case and baseurl= http://localhost/Joomla1.5.7)

    Architecture

    3rd Party

    • I use antelope for some advance ANT operations: substring, indexof, loop
    • I use selenium java server on port 4444 (default)

    Cluster

    All cluster operations are in cluster.xml these basic functions are

    • cluster.init
      • cluster.remove        remove all instances of Joomla! in checkout directory
      • joomla.unpack.all    unpack all Joomla! versions that are checked in another SVN root
      • joomla.install.all      run the same selenium test case joomla.install.html on all Joomla! instance
      • joomla.remove.all.installation   remove all Joomla! installation directories
      • joomla.check.all     check all Joomla! installations for correctness
    • cluster.start
    • cluster.remove
    • cluster.stop

    Joomla!

    All Joomla specific operations are in joomla.library.xml

    • Unpack a Joomla! version
    • Remove the installation directory form a version
    • Apply a selenium test suite install.joomla.html that can use the regular Joomla! installer
    • Is also able to do all of the above on all Joomla! versions found (regular expression) in checkout directory

    Selenium

    • All selenium operations are in selenium.xml
    • All test suite and test cases are in /selenium/joomla/

    PHPUnit

    All PHPUnit operation are in phpunit.xml

    Settings

    Settings are in setEnv.xml, in future I will lazy load a file if it exist as environment variable

     

    If you know ANT, the code is quite readable...

  • TeamCity 4.5 up and running

    teamcity512

    Today JetBrains announces the public availability of TeamCity 4.5, it's award-winning distributed build management and continuous integration tool.
    With TeamCity, you can set up a build server for your projects within minutes and enjoy out of the box continuous unit testing, code quality analysis, and early reporting on build problems - even without leaving your favorite IDE.
    TeamCity offers a gentle learning curve, so you can quickly improve your release management practices by gradually adopting its advanced features and capabilities.

    I did upgrade my version of TeamCity at teamcity.waltercedric.com to the latest version, aka 4.5

    What’s new?

    This release brings many usability improvements, updated IDE plugin with some new VCSs supported, plus exciting enterprise-level features, not found in previous versions:

    • User Groups are now supported when defining user roles and notifications
    • Extended LDAP support with automatic user profile synchronization
    • Multiple UI improvements, including tests grouping and project-wide test details, problematic tests, and change log
    • You can now assign responsibility for broken builds
    • In addition to traditional test results handling, TeamCity is now able to parse raw XML reports from ANT's JUnit tasks, NUnit, Surefire, PMD, and FindBugs
    • Mono framework is now automatically detected and can be used as a .NET platform for continuous builds
    • Improved IDE integrations:
    • Eclipse: CVS is now supported for Remote Run and Pre-tested Commit; added support for Perforce plugin version 2008.2
    • Visual Studio: Remote Run and Pre-tested Commit now work with Perforce

    To see What's New in TeamCity 4.5, just visit this page

    Upgrade from TeamCity 4.0.2 to TeamCity 4.5

    The upgrade went flawlessly

    # /home/teamcity/Teamcity/bin/runAll.sh stop kill

    Quick backup of file system

    # cp –pir /home/teamcity/ /home/teamcity.4.0.2

    Quick backup of database

    # mysqldump –uUser -pPassword Schema > /home/teamcity.4.0.2/teamcity.sql

    Get the latest version

    # wgethttp://download.jetbrains.com/teamcity/TeamCity-4.5.tar.gz

    Delete the old installation (was getting some spring error by just overwriting the previous install)

    # rm –rf /home/teamcity/Teamcity

    unpack

    # tar xvf TeamCity-4.5.tar.gz .

    I did copy some needed stuff from previous installations like server.xml (conf/server.xml) and MYSQL driver (WEB-INF/lib),

    Restart and voila TeamCity 4.5 is up and running!

  • TeamCity 5.0 now available

    Amazon EC2 and Virtual Agents

    Take avdantage of cloud computing with TeamCity by putting Build Agents on Amazon EC2. After your Build Agents' AMIs (Amazon Machine Images) are registered, the TeamCity server will account for those virtual machines in its build scheduling and will automatically do a suspend/resume job on them, based on the load and build queue status.

    Better Support for Maven

    Version 5.0 has many Maven support improvements under the hood. These include the creation of Maven-based build configurations (only the URL to the POM file is needed), the visualization of various project data gathered from POM, and a new type of dependency trigger - Maven artifact trigger. This type of trigger starts a build each time the specified Maven artifact changes.

    Build Configuration Templates

    Templates can save a lot of manual work when you need to define a number of duplicate or very similar build configurations. Now you can create a template with the common (shared) settings, and then inherit any number of build configs from this template. You can also extract a template from any of your existing build configurations.

    Issue Tracker Integration

    How often do you see comments like this: "TW-3431 fixed"? How about a little more detail on the issue at hand? The issue tracker integration provides an in-place bug title, direct navigation to the issue, and other productivity goodies. It is currently available for Jira, Bugzilla and JetBrains YouTrack out of the box, but can easily be extended to other systems via plugins

    Command Line Tool for Personal Builds

    Previously, if you weren't using Eclipse, IntelliJ IDEA or Visual Studio, you were out of luck for missing one of the most important TeamCity features - Remote Run and Pre-Tested Commits. But things have changed. As the name suggests, now you can initiate the building of your local changes remotely on the server - from the local command line. No IDEs required.

    Many Other Improvements

    Including:

    • Per-test responsibilities
    • Build artifacts promotion
    • Dedicated change details/status page
    • Incremental update of the build log
    • Project archiving
    • Backup & Restore for easier TeamCity maintenance. Backup can be started from the UI or from the command line; Restore and Migration can be started from the command line only. Command line backup does not require stopping TeamCity if an external database is used.
    • Audit of user actions
    • Coverage support for .NET & Java. Based on IntelliJ IDEA coverage engine for Java and on NCover
      (ver 1, ver 3) or PartCover for .NET.
    • Git & Mercurial support now bundled with TeamCity
    • Remote Run for Git from IntelliJ IDEA & Eclipse
    • Support for Cucumber, Shoulda, Test-Spec in Rake runner
    • TeamCity IntelliJ IDEA plugin now works with IntelliJ IDEA Community Edition and RubyMine.

      Download TeamCity 5.0 (free 60-day trial). See also complete release notes

  • TeamCity remote debug your Maven build

    apache_maven

    How to remote debug test cases

    Change the Team city project configuration by adding a -Dmaven.surefire.debug to Maven runner in Additional "Maven command line parameters"

    maven.remote.debug.testcases

    Now when test cases will be executed by maven surefire plugin, the build will wait for a remote debugging application to pick it up on port 5005 and this for EVERY MODULES
    meaning:  If you have 5 Maven modules (= java projects) with test cases maven surefire will request 5 times you to connect with remote debugging to your build server.

    Create a Remote Java Application launcher you'll also share in one eclipse project:

    maven.remote.debug.launcher

     

    Don't forget to remove the -D variable or your daily build may wait for a remote debug connection! or create a special build configuration of your project targeted for debugging purpose.

    Remote debugging Maven plugin

    put into "JVM command line parameters:" these settings:

    -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5006

    debug.maven.in.teamcity

    Create a Remote Java Application launcher you'll also share in one eclipse project that will connect to the port 5006.

  • TeamCity updated to 5.1.1 and a Joomla! module soon available :-)

    I did upgrade from TeamCity 5.0.1 to TeamCity 5.1.1 In no time, just drop the war in my Tomcat container :-)

    At this address you will always find the latest snaphosts of some (I would need 44 builds to display them all!) of my PHP projects for Joomla!

    http://teamcity.waltercedric.com

    I will also upgrade tomorrow early morning our TeamCity Licensed server at www.Innoveo.com .

    Continuous build for Joomla!

    By the way I have still to deliver (HERE and HERE) what i promised on the Joomla! development group, aka a full solution for developing Joomla! using Maven for PHP and Teamcity/Bamboo/Husdon. You can see the documentation I made in my WIKI (work in progress)

    TeamCity integration in Joomla!

    At the same time, Joomla! users that are using TeamCity, search no more, I will be providing you the first GPL module for Joomla! that let you display in your site the status of all your builds/projects! The code is heavily using the REST API with PHP CURL and a bit of XML parsing. If you have any requirements, it is still time to drop me an email ;-)

  • Tux Droid

    tux-droid-linux-companionTux Droid is a Linux wireless Tux mascot (210mm x 180mm x 140mm - with lowered wings) with a programmable interface, allowing it to announce events by its gestures and by ALSA driven sound. The events are detected by specific gadgets, which are handled by the Tux Gadget Manager.

    The Tux Droid supports Linux kernel 2.4 or later and needs a 800 MHz CPU and 128 MB RAM. It communicates by infrared to the USB port (1.1 or 2.0) and for media detection it needs an internet connection. The mascot is driven by Atmel AVR RISC microcontrollers.

    The mascot comes with a microphone and an infrared receiver, to perform a 2.4 GHz wireless full-duplex digital link between the USB dongle.

    The Tux Droid also has a light sensor and a push button on top of the head. Its gestures cover e.g. eye- and wing-movements, while switch sensors in both wings are triggered by pushing the wings. For its sound output there is a volume control wheel to control a speaker and a 3.5mm stereo audio socket for audio out.

    [WikiPedia]

     

     

    My tux Droid has arrived 2 days ago.

    Shortly here is a list of what I do not like

    • Noisy gearbox, compare to the Nabaztag, there is world in between.
    • 2.4GHZ but no WIFI, so it need always a running server. Hope the would develop a WIFI
    • Use Acapela voice engine, which should be the best on market, but voices are not really as clear as on the Nabaztag.
    • Less gadget, more in the 20 range.

    And what I like a lot

    • Open source hardware and software,
    • Many programming language: Python, Java,
    • Easy to program gadget,
    • A lot more response feedback: yes, mouth, flaps, rotation.
    • Very good wiki, and online documentation
    • It look like TUX :-)

    I did develop a Tux Droid plugin for TeamCity which is not far away from running and be distributed under GPL v3

    Download the latest software go to the kysoh website.
    Developers documentation visit our wiki.
    For the forum go here.
    For the trackers go here.
    And for the community website go here.

  • Tux Droid Plugin for Jet Brains TeamCity 4.0

    tux-droid-linux-companionTux Droid is a Linux wireless Tux mascot (210mm x 180mm x 140mm - with lowered wings) with a programmable interface, allowing it to announce events by its gestures and by ALSA driven sound. The events are detected by specific gadgets, which are handled by the Tux Gadget Manager. The Tux Droid supports Linux kernel 2.4 or later and needs a 800 MHz CPU and 128 MB RAM. It communicates by infrared to the USB port (1.1 or 2.0) and for media detection it needs an internet connection. The mascot is driven by Atmel AVR RISC microcontrollers. From http://en.wikipedia.org/wiki/Tux_Droid

    TeamCity is a Java-based build management and continuous integration server from JetBrains, creators of IntelliJ IDEA and ReSharper.

     

     

     

     

    Tux Droid Plugin

    With this plugin you can notify your Tux Droid with your TeamCity build results.

    After login you can find the new notificator Tux Droid in your notification settings. You need your Tux Droid server IP and port  to connect your Tux to TeamCity. After you have saved the parameters you can configure your individual notification settings for just all projects/specific projects/build failures/...

    Each registered User can customize Tux Droid messages or choose different Attitunes.

    Attitunes are robot animations you can create combining movement, sound, Text-To-Speech and fun. From functional to fun-ctional. Make Tux Droid sing 'Dancing in the rain' or make him cry when you didn't receive any new E-mail. 

    Some quick info

    • Code is GPL v3.0
    • It is version 0.0.1,
    • It is build with Maven 2.0, eclipse project to be available soon.

    For now just visit the official home of the Tux Droid plugin for TeamCity

    http://tuxdroid.waltercedric.com

    Documentation is in my Wiki:

    http://wiki.waltercedric.com

    Moreover, I’ve decided to install and maintain:

  • My own maven repository http://maven.waltercedric.com
  • My own build server  http://continuousbuildserver.waltercedric.com/

  • Update JetBrains TeamCity in one click

    penguin  A small script developed to upgrade TeamCity with no or less effort! a very simple script, easily extensible.

    TeamCity is a continuous integration and build management system. With TeamCity, you can set up a build server within minutes and enjoy out of the box continuous unit testing, code quality analysis, and early reporting on build problems — even without leaving your favorite IDE. TeamCity offers a gentle learning curve, so you can quickly improve your release management practices by gradually adopting its advanced features and capabilities.”

     

     

    Limitations

    This script only work if

    • You run TeamCity using the standalone package provided at http://www.jetbrains.com/teamcity/download/index.html#linux under Linux
    • You use MYSQL as an external DB
    • You must run it as root (because getting the latest software with wget may not otherwise be possible)
    • This script is not endorsed by JetBrains nor official.
    • GNU/GPL version 3, use at your own risk, provided as IS

    It must be safe to use, as

    • It backup first your database
    • it archive your existing TeamCity installation with a timestamp
    • it also copy any customizations (server.xml, agent build.properties) you have made to the new installation
    • It download automatically and extract teamcity

    Usage

    # ./updateTeamCity.sh newVersionNumber installPathHome

    example:

    /home/teamcity > ./updateTeamCity.sh 4.5.4 /home/teamcity

    Output of the script

    stopping current TeamCity at /home/teamcity/TeamCity
    done!
    Read database settings from database.properties
    Moving current TeamCity /home/teamcity/TeamCity to /home/teamcity/TeamCity.20093830
    done!
    Saving database to file system at
    /home/teamcity/TeamCity.20093830/teamcity.sql
    done!
    getting the latest version at http://download.jetbrains.com/teamcity/TeamCity-4.5.4.tar.gz
    done!
    Unpacking new version
    done! 
    Copying previous customizations
    done!
    Copying required 3rd party librairies
    done!
    Copying local agent build.properties to new TeamCity
    done!
    Starting new TeamCity
    done!

    Don’t expect more at the moment, it has been created to make the update of TeamCity faster for ME now. Feedback is appreciated, and improvement/ideas are always welcomed.

    GET THE SCRIPT updateTeamCity HERE

  • Update Maven pom version on GIT checkout in TeamCity

    Here is a solution to the following problems

    • Deriving #Maven artifact version from #GIT branch,
    • Update pom version on GIT checkout automatically,
    • Add the ability to use Pull request with Apache #Maven.

    You have a workflow requirement that require you to have the artifact version of a module externally defined from the current branch in GIT.

    For example

    You want to start working on a new feature branch “feature-memory-improvement”, so you branch from master a new branch named feature/feature-memory-improvement

    Having unique snapshot is a something you need to share your code using a #Maven repository, so you may want to have into the branch all pom.xml version changed to

    <version>FEATURE-MEMORY-IMPROVEMENT-SNAPHOTS</version>

    changing all your pom.xml and doing a technical commit  will create merge conflicts when using pull request!

    One solution, while not perfect is to do the following:  You can add a separate execution to run a goal which will change the version of the POM automatically in the #Maven reactor. This small script will do it¨

  • When Joomla! meet Maven for PHP

    apache_maven

    joomla logo vert color

      I will quickly describe in this post how you can code your component/plugins/modules against a specific version of Joomla! with no effort using Maven for PHP.

    The trick I made is to package the whole Joomla distribution in a jar file, and add a Maven dependency to the project. As Joomla, PHPUnit, or any other 3rd party is not a set of class packaged in a jar, the brilliant idea of Maven for PHP is to unpack all these dependencies in the target/phpinc directory automatically AND concatenating this directory to the php setting include_path during compile and test phase. You can do the same using the command line by running:

    php -d include_path=

    That’s also why, It is very important to not put anything from this directory under version control (add a cvs.ignore or svn properties svn:ignore) as it contains dependencies that have no reason to be put under version control in Maven paradigm (they have to reside in a Maven repository)

    So I create a new zip file joomla-1.5.10.jar with a directory inside named ‘Joomla’ that contains a standard binary distribution…this jar is published in artifactory with a groupid org.joomla (arbitrary chosen by me)

    Artifactory will generate a default pom.xml. Normally it should contains more meta data and be created by the Joomla team itself. (if they ever support Maven)

    <?xml version="1.0" encoding="UTF-8"?>
    <project>
      <modelVersion>4.0.0</modelVersion>
      <groupId>org.joomla</groupId>
      <artifactId>joomla</artifactId>
      <version>1.5.10</version>
      <description></description>
    </project>

    The artifact is deployed and viewable at http://maven.waltercedric.com/artifactory/webapp/browserepo.html

    joomla.in.artifactory

    With the code below I can bind my component to a new version or old version of Joomla just by modifying the component pom.xml. Note the version number 1.5.10 (the latest version of Joomla! to date)

    <dependency>
        <groupId>org.joomla</groupId>
        <artifactId>joomla</artifactId>
        <version>1.5.10</version> 
    </dependency>

    Test cases classes are directly in the eclipse project under src/test/php (this directory is created by the archetype php5-lib-archetype), In bold, this is how you can import the whole Joomla! framework!

    require_once ('PHPUnit/Framework.php');
    require_once ('Joomla/index.php');

    //I load the PHP class that I would like to test
    require_once ('admin/class/logUtils.php');

    class LogUtilsTest extends PHPUnit_Framework_TestCase {

    //a poor PHPUNit testcase
    function testisAnEmail() {

            $LogUtils = new LogUtils();
            $this->assertEquals(true, $LogUtils->isAnEmail("This email address is being protected from spambots. You need JavaScript enabled to view it."));
            $this->assertEquals(false, $LogUtils->isAnEmail("cedric.walterwaltercedric.com"));
            $this->assertEquals(true, $LogUtils->isAnEmail("This email address is being protected from spambots. You need JavaScript enabled to view it."));

        }
    }

    More to come soon.