XenServer automated testing laboratory and orchestration - the introduction of XenRT
Introduction
XenRT , the automated test system Citrix XenServer, has reached an important milestone this month. The 500,000th test job was executed. And that in the 8 years since XenRT began life testing the Xen hypervisor and XenSource XenEnterprise product in the summer of 05.
XenRT is synonymous with Xen regression testing , and was designed and built by a team led by James Bulpin, then head of QA XenSource (Citrix XenServer later QA) and currently senior director of technology for Citrix XenServer.
Over the years XenRT evolved from an automated test system comprising tens of automated test cases running on some servers, focusing much on the Xen functionality regression test core Hypervisor, a highly sophisticated platform test-as-a-service. The current incarnation of XenRT within Citrix performs testwork at the request of the XenServer engineering group. Tests on a large laboratory distributed with a large variety of hardware types. A team of twenty-five developers in the UK and India spend most of their time either maintaining and XenRT operation or development of new cases of automated test for XenRT. However, a mark of his success as a test platform as-a-service is that it is common for test cases to be developed and managed by the wider engineering community XenServer within Citrix . Each case additional test, once developed, is added to the ever growing inventory of test cases available to the team XenServer.
Any engineer can use XenRT to ask a set of test cases (ranging from a single test case to a subset of thousands available) to be performed, and that the results be returned by email to them. Each application is under XenRT after which it is allocated job ID and is fully traceable at runtime.
In August 2013, we celebrated the XenRT 500,000 jobs!
architecture
XenRT the core is the planner. This is a piece of software (like everyone XenRT code, written in Python) that schedules user requests, map them to the available laboratory equipment. Some jobs will require specific hardware - eg CPU type or specific storage array. In the general case of jobs will be scheduled to run on any hardware that is initially available that meets the constraints specified in the job. A beneficial side effect of this is that a given test case, unless it has very specific constraints, will, on several tracks, be exercised on different server types and combinations of materials, providing a degree of testing equipment diversity. This helps to address one of many challenges to test an operating system such as XenServer -. That validate its operation through many types of materials taken different load
What we call the "laboratory XenRT" is actually several physical laboratories distributed across several geos. The laboratory is also logically divided into "XenRT sites," most sites including 16 servers, including some as much as 48. Each site is managed by a dedicated server called "XenRT Controller". As expected, a work is handed to relevant XenRT controller. When he performs the first pass through a phase of installation where he orchestrates the test setup required on the server or host pool - this usually involves installing the required version of XenServer, installing the required guests and configuration network and storage. Once implemented, the tests themselves are executed, the test logs and results are stored in a database and defects are self-generated in the tracking system XenServer defects (including automated deduplication) .
Some jobs are requests by individual developers and test engineers to perform a particular test. Some jobs are suites of standard regression tests, with hundreds or even thousands of test cases. Many of these suites are continuously operated to provide for continuous quality of the mainline code branch XenServer. The net effect is that the laboratory XenRT has a very strong, effective in using 24 × 7, 365 days a year.
As well as functional test cases, XenRT has suites of test cases which measure the performance characteristics of XenServer, the stability test and large scale performance features (thousands of customers crossing large XenServer pools), this interoperability test of XenServer with other Citrix products such as XenDesktop, NetScaler and PVS and test XenServer in continuous use in the long term. The codebase XenRT is object oriented with lots of reusable code library for common tasks in the orchestration and the XenServer operating environments. This leads to an ever decreasing cost of developing test cases
Summary
XenRT is a key tool for engineering XenServer -. It provides the XenServer team with a continuous integration capacity; it ensures rigorous regression testing features functional and non-functional systems; it is 100% automated; he gets the maximum expensive capital equipment ensuring a very high use of laboratory; it provides a rich set of searchable newspaper and measures; it provides laboratory management capabilities and rich orchestration. All this allows the team to XenServer completely releases a timescale much shorter and more efficiently than would be the case. Achieving a similar level of test coverage only by manual testing would be prohibitive.
And then?
XenRT remain a key element of quality assurance strategy for Citrix XenServer. However, the next step on the path XenRT is to allow the community to benefit xenserver.org the same tests as-a-service features that Citrix XenServer team benefits. Beware a forthcoming blog on xenserver.org giving more details on this exciting development!
So what the groups are, suites and families? 
The Society of Exploration Geophysicists (SEG) International Exhibition is the largest oil , the world's energy and mineral exhibition showcase the latest technology for use in mining and related industries. 
