Google faces the world’s largest, toughest software testing challenges: To get it right, Google is pioneering the future of testing and automation.
You also need to get it right. Learn from Google. At Google, ensuring test coverage for code is mostly the job of the developers who authored that code. There are no dedicated testing departments, either. Instead, software testing falls under the jurisdiction of the branch known as Engineering Productivity (Eng Prod).
Test exists within a Focus Area called Engineering Productivity. Eng Prod owns any number of horizontal and vertical engineering disciplines, Test is the biggest. In a nutshell, Eng Prod is made of:
1. A product team that produces internal and open source productivity tools that are consumed by all walks of engineers across the company. They build and maintain code analyzers, IDEs, test case management systems, automated testing tools, build systems, source control systems, code review schedulers, bug databases… The idea is to make the tools that make engineers more productive. Tools are a very large part of the strategic goal of prevention over detection.
2. A services team that provides expertise to Google product teams on a wide array of topics including tools, documentation, testing, release management, training and so forth. Their expertise covers reliability, security, internationalization, etc., as well as product-specific functional issues that Google product teams might face. Every other FA has access to Eng Prod expertise.
3. Embedded engineers that are effectively loaned out to Google product teams on an as-needed basis. Some of these engineers might sit with the same product teams for years, others cycle through teams wherever they are needed most. Google encourages all its engineers to change product teams often to stay fresh, engaged and objective. Testers are no different but the cadence of changing teams is left to the individual.
Google has two testing-related roles: Test Engineer and Software Engineer, Tools and Infrastructure.
Test Engineers (TEs)
Test engineers’ main focus is to keep Google’s continuous automation testing infrastructure efficient and effective. The role of TEs encompasses knowledge sharing, fostering best practices, and tools development. The goals set before TEs include:
- Test prioritization (i.e. determining what features need better test coverage).
- Writing test scripts and building testable user journeys that developers can use to test their own code.
- Development of in-depth QA knowledge and expertise in product teams.
- Finding weak spots in the product codebase.
- Finding new ways to break software and identify bugs
- Fostering communication between product stakeholders to ensure successful releases.
Software Engineers, Tools and Infrastructure (SETIs)
SETIs are engineers building, orchestrating, and continually improving tools, frameworks, and packages used in software development and testing. Originally known as Software Engineers in Test, this role expanded beyond product testing. To mark this move, Google rebranded the SET role to Software Engineers, Tools and Infrastructure (SETIs) in 2016.
The key responsibilities of SETIs include:
- Development of tools for automated testing, software development, performance monitoring, etc.
- Collaboration with TEs aimed to streamline communication and foster the adoption of best practices across product teams at Google.
- Collaboration with other specialists on conferences (e.g. Google Test Automation Conference).
Automation testing tools at Google
Knowing that Google’s testers are really software engineers building test frameworks and tools, what are these frameworks and tools? Here’s a short list of the well-known testing solutions built or influenced by Google:
Selenium. No, Google didn’t build Selenium, but Jason Huggins (the creator of Selenium) worked at Google in 2007 on Selenium RC. Besides, it was the Google Test Automation Conference 2009 where Google and ThoughtWorks agreed to merge Selenium and WebDriver into Selenium 2.0. What followed is now history.
Protractor. Originally developed for end-to-end testing of Angular apps, Protractor is one of the most popular automation frameworks. Needless to say, engineers from Google use Protractor for many products and play an important part in the development of this framework.
Karma. The spectacular test runner for JavaScript is the brainchild of the AngularJS team at Google. Also, the original name of this test runner was Testacular.
Espresso UI is a test framework and a record-playback tool for Android.
EarlGrey. In addition to UI testing for web application and Android, Google taps into functional UI testing for iOS with EarlGrey. At Google, this framework is integral to the UI testing of Google apps for iOS, including YouTube, Play Music, Google Calendar, Google Translate, etc.
GoogleTest. The products that use this C++ test framework are the Chrome browser, Chrome OC, and the computer vision library OpenCV.
Google Test Case Manager is a test management software that Search giant uses internally.
OSS-Fuzz is Google’s solution for the fuzz testing of open-source software.
Martian proxy is a library for building programmable HTTP proxies for testing purposes.
Manual testing at Google
- Google uses manual tests to identify non-trivial problems with their products (i.e. exploratory testing).
- Manual testing is a “Plan B” method for web testing when HTML lacks proper tagging (e.g. IDs). Developers typically resort to manual testing if adding (or retrofitting) proper tagging isn’t an option, or if the structure of a web app is unstable. Cases like this mostly occur in the early stages of product development.
- Judging by LinkedIn, Google outsources parts of its testing efforts to overseas development shops. In some cases, outsourced jobs involve manual testing.
Google’s answer is to split the role. They solve this problem by having two types of testing roles at Google to solve two very different testing problems.
