During the last few weeks we have gotten more questions on our build cycle; specifically, how do we build? How much work can possibly go into a “remaster” of Ubuntu? Well despite what you have read and what people say there is a lot more that goes into Linspire and Freespire than remastering. This is what happens when we build the distribution(s) :
Hardware
The build hardware is not unique : a Dell PowerEdge T630
Hard Drive: 1 TB
RAM: 32 gb
Video Card: Matrox G200
OS: Build 14210 which is our own Gentoo based system
Other Software installed: VirtualBox
In the beginning
Initially we download the Xubuntu or Ubuntu ISO; then everything is stripped out. No web browser, office suite, multimedia apps : no installed applications whatsoever. We add the common tools, package manager, text editor, build-essentials, and a few graphics elements such as themes, icons and wallpapers. This build produces an ISO image that’s 680 mb and runs in 260 mb of RAM.
The GUI
Unity was originally going to be the GUI. We liked Unity and its ergonomics scratched every itch we had. After Canonical announced that it was discontinuing Unity development, focus switched to XFCE as the default GUI. We then decided on panel layouts and the look that we wanted to achieve.
Bug research
We then spend about 6 weeks looking at Launchpad and bug reports from our own customer base; we look into them, try to replicate them, and fix what we can. These results and any code we generate are submitted back to the individual communities. We track and document these bugs and during the testing phase we check to make sure that it was resolved or if the resolved bug breaks something else (regression testing).
Building the bytes
At this time we start building our software. We look at the Ubuntu mainline kernels available and select the one that’s most compatible with packages and custom drivers that we want to include. At this point, we start building the base kernel that will eventually become Freespire / Linspire : 1 image installed it in its own VM, software being built on this image based on the custom kernel. This process takes about 2 weeks.
Starting to come together
After we have 1). built our core GUI and 2). made kernel choices, we then generate test images. The ISO generated here is about 1.1 gb and runs in 320 mb of RAM. At this point the development diverges between Freespire and Linspire. The teams take this image and test it for a few weeks to make sure it operates correctly : no unusual bugs, benchmarking, etc. This same image is also installed on our testbed of 12 different machines selected from representative vendors to make sure individual hardware builds don’t experience major issues. These vary - server class systems, desktops and some test boards supplied by partners.
Moving in day
After that process then comes the installation of packages. At this point we start installing things like web browsers (Chromium in Freespire, Google Chrome in Linspire), office suites, games, multimedia, codecs and other software. Some of it’s packaged, some is not and is injected directly into the system itself. As the libre build, Freespire is lighter and includes fewer components and is 1.4 gb; being more full-featured and containing more proprietary drivers and components, Linspire weighs in at a slightly heavier 1.9 gb.
At this point both OSes are feature-frozen; meaning, afterwards there should be no major changes to the user experience.
Test, Test, Test
Both builds are very heavily tested; which takes time because of the limited selection of machines available for our testbed. At this point many of the developers have upgraded to this new system build.
Let’s go outside
We ship our new builds out to the closed beta program for Freespire and ship Linspire to commercial beta testers and insiders. Customers and partners can also test and perform their own internal certifications. The teams use this time to collect bug reports and document problems that have been experienced. This is an extremely long process, generally taking about 3 to 4 months. At this stage we contact OEM’s and ship them the software so that they can test on their systems for future QA.
Fix er up
We take the information that was relayed to us by testers and we fix what problems we can and we document the rest for fix later. What issues are chosen for repair? We prioritize the problems that we can replicate and that affected the most people. As other developers can attest to we all suffer the the “it worked last night “ phenomena. This step can take quite a bit of time depending on the number of issues that need to be fixed.
QA (Quality Assurance)
At this point we arrive at Quality Assurance testing. We have a specific group that uses the products and gives their feedback. We use this feedback to improve the products and if we can include it in this release then we do it, if not and if it's something that will break the workflow of others we list it for possible inclusion in future releases. Quality Assurance testing is a complex stage as sometimes we disagree with the testers and at this point we try to find compromises
Public Beta testing
We have arrived at public beta testing phase when all that has preceded has been said and done. Freespire is released to the public for testing, Linspire to our insiders, and in both cases we collect their feedback. We also send it to partners, resellers and customers who need to certify it against their own products, services and applications. Public Beta testing usually takes a couple of months and there are several revisions released (as necessary) until we’re satisfied and can announce a Release Candidate.
Go forth, my son
After all the public Betas have been released and feedback received (at this point, usually favorable because of the rigorous internal beta testing and QA) the Linspire team has a general or Gold release; this is marketed as the finished product, and shipped to customers and users. Freespire’s Gold release is delayed for public download for two weeks and then PC/OS notifies the press and other marketing partners. These general releases are followed by what we call incremental releases.
Incremental updates
Incremental updates don't go through the same rigorous testing and development phases as our general / major releases. With incremental releases we apply all application updates and security updates and perform a limited round of public beta testing where users can provide feedback and bugs reports.
Conclusion
So there you go. This is how our products are built. A little more involved than a simple remaster, don’t you think? Black Lab Linux undergoes a very similar development process, albeit longer, due to its use in high performance and development workloads. Right now, for the next versions of Linspire and Freespire, 8 and 4 respectively, we are in the very first stages of development and building the primary image. Our year is generally very busy and things happen to lengthen / shorten the development cycle : serious bugs and major issues, less serious issues and minor bugs. This is what’s involved in building a contemporary Linux distribution.
