It’s always good to have someone experienced by your side, telling you what’s right and what’s not. What’s true in life is also true in software development, but here coding standards play that role. They make sure your code follows the best industry practices and is foolproof against attacks. But as is the human nature, mistakes can be made – and even more so when working to strict deadlines. For example, developers seldom have time to confirm code compliance when trying to finish their features.
While there are static code analysers that can help with checking compliance, SonarQube stands out among them because of its enterprise-level robustness, user-centric design and its very innovative features. In this tutorial we will have a closer look at how to set up SonarQube to analyse a Python project, followed by information on how to add customised rules for code analysis. Later we will have a quick look on integration with Jenkins to make the entire process automated as well.
Step 01 Understanding SonarQube architecture
Before we get started with installation, let’s try and understand SonarQube architecture and the various components. SonarQube has three main components: the SonarQube source and web server, the database and the analysers. It’s the analyser that actually interacts with your source code and updates the database (note that there may be more than one analyser involved, but more on that later). The web-based frontend then renders the analysis results to you in a comprehensible format. These components work independently yet in close collaboration, making SonarQube robust and scalable. It is even possible to install these components on different servers to balance load.
Step 02 Database installation
The first step during installation of SonarQube is to install the database. SonarQube supports almost all the major databases out there, including MySQL, Oracle, PostgreSQL and Microsoft SQL server. In this tutorial however we will use MySQL. You can install the standalone version of MySQL or use the open source web-hosting stacks such as LAMP/MAMP/WAMP servers (note that this approach may not be the best for enterprise installations due to performance issues, as we would be using only MySQL and not the web server included in the stack). Just make sure you know the port your MySQL is running on. After the installation, create a user with root privileges to be used by SonarQube and database ‘sonar’ to store the data generated by analysers.
Step 03 Server and SonarQube installation
Next step is to install the web server. Unzip the package you have downloaded from SonarQube website and navigate to the ‘sonar. properties’ file in the ‘conf’ folder. Uncomment the MySQL database connection strings, while making sure all other connection strings are commented. Then, change the port in the MySQL connection string to the port your MySQL is running and check if the database name is the same as the one you’ve created. Also, update the credentials for the user you created in the previous step. As you would have understood by now, the aim here is to enable a connection between database and SonarQube. You can choose to modify the values in properties file, but make sure the database corresponds to the changes.
Now navigate to the ‘bin’ folder and then to the folder corresponding to your operating system before executing the ‘sonar.sh’/‘sonar. bat’ file with ‘start’ argument. This starts your webserver, and if everything is fine you can open SonarQube home on your browser at http://0.0.0.0:9000, which is the default address and port and can be changed from the ‘sonar.properties’ file). Now download the Python plug-in for SonarQube and place the JAR file to the path
Step 04 Analysers
Analysers serve as the bridge between the project to be analysed and the SonarQube database. So, naturally there are different analysers available for different project types, eg Maven analyser for Maven projects, SonarQube Ant task for projects built with Ant, and so on. We plan to analyse a non- Maven Python project here, so we will install the SonarQube Runner. Download and unzip it inside the folder where your SonarQube resides – this is to make sure all SonarQube files are in one place, but you can put it in any location you like – then navigate to the
Step 05 Analyse the project
To analyse your project, navigate to your project folder and create a file called ‘sonar- project.properties’ at a convenient location inside, with the below contents:
------------------------------------------- # Required metadata sonar.projectKey=org.codehaus.sonar:example-python-sonar-runner sonar.projectName=Python project analyzed with the SonarQube Runner sonar.projectVersion=1.0 # Comma-separated paths to directories # with sources. Starting SonarQube 4.2 # this is optional. If not set, SonarQube # starts looking for the source code from # the directory containing sonar-project. # properties file. sonar.sources=src # Language sonar.language=py # Encoding of the source files sonar.sourceEncoding=UTF-8 -------------------------------------------
This is the standard SonarQube format for project properties file. Once you create this file with details relevant to your project, navigate to the folder containing this file via command prompt and run the command sonar-runner (this should work if you have set the path properly in the previous step). Once run successfully, the analysis of this project is stored to the database via the SonarQube runner, which can be then seen using the web interface.
We have taken the sample Python project available on the SonarQube website for the purposes of demonstration. You can use the same steps to analyse your own project.
Step 06 SonarQube GUI
Now that the backend is ready, let’s deep dive into the UI. Default credentials are admin, admin. After login, SonarQube greets you with the default dashboard (also called Home) and default widgets. Don’t forget to change the password by accessing your profile on the top-right corner. Configuration of homepage elements is quite simple; you can easily configure the widgets for the default dashboard or even add new dashboards. In the top-right below the search box, you have the ‘Configure widgets’ and ‘Manage dashboards’ options available to help you do so. The ‘Project’ link on the top menu shows the name of project most recently active and gives the option to view all the projects. In our case there is only the one project.
Step 07 Metrics
How would you go about tracking your project’s status? Probable answers are on the basis of code coverage, code complexity, documentation, bugs and so on. So, basically, tracking can be done with some metric. Let’s see how SonarQube lets you do this easily and efficiently, with an example. Suppose you want to see the files in the project with comment lines more than 30. Simply click on the ‘Measures’ link on the top menu, fill ‘Files’ in the text box, and then select ‘Metric’ in the ‘More Criteria’ field. Click on the ‘Metric’ again to display the list of all the metrics available. Now select ‘Comment Lines’ under the ‘Documentation’ header and select the value as greater than 30. This lists out the files with more than 30 lines of comments. To save yourself from having to do these steps again, just click on the ‘Save as’ link in the top-right corner.
Step 08 Issue tracking
All the instances of compliance rule violation reported by the SonarQube run are logged as issues. Click on the ‘Issues’ link in the top menu to view the list of all the issues reported. To look at issues with project perspective, you can first go to the project you want to see the issues for and then click on the ‘Issues Drilldown’ link on the left menu bar. Here you get an overview of issue severity, rules broken, related files and so on. This can help a lot when it comes to identifying whether there is an issue reported by the tool that you don’t want to consider now. SonarQube lets you manage issues just as any bug-tracking tool would do. You can assign them to different users and resolve them after they are fixed in source code. Note that you are always allowed to override the severity assigned by default and even report issues as false positives.
Step 09 Add your own code compliance rules
SonarQube allows you to add custom rules as well. The rules can be written in Java via a plug-in, or can be added via the web interface as XPath rules. As per the SonarQube website, the XPath option will be eventually dropped, but since it allows adding rules a little bit faster, we’re going to make use of it here. Go to the quality profile and then click on the profile you want to add the rule in. Search for the ‘xpath’ name/key with activation set to ‘Any’. This will bring you the XPath template to add rules. Click on the template and you will see the link to ‘Copy rule’ below. Click on it and then you get to the ‘new rule’ tab. Here you can enter your XPath rule in the field ‘xpathQuery’. If everything is fine, next time you run sonar-runner on the project, you can see issues related to new rule also being logged. Yes, the prerequisite to add rules is knowledge of XPath, but you can take a crash course at www.w3schools.com/xpath.
Step 10 Jenkins and SonarQube
As some of you may be aware, Jenkins is a continuous integration server, which lets you build and even test your project continuously without any manual interference. Once you configure and start the server, it takes code from SCM, builds and even runs the tests and sends you updates at regular intervals. Now SonarQube has a plug-in available, which enables Jenkins to trigger the analysis. With this, the analysis of your project becomes automated, and you get regular updates about the project while doing other important work! Note that the plug-in has to be installed to Jenkins, so that Jenkins can see the SonarQube servers and trigger the analysis. Read more about the configuration on the SonarQube docs page.