Product Evaluations

How we Evaluate IT products

When it comes to testing enterprise IT products, MagLar stands head and shoulders above the rest, with in-depth hands on testing of the leading IT solutions . You might know us best for our Evaluations, which enable you to compare the features and performance of many vendors at a glance.

To get the most for your IT budget, here is an overview of our unique hands-on testing process.

What we test

MagLar currently tests about 100 enterprise IT products every year, in seven or so categories. We cover everything from data protection and CMDBs to virtualization, ITSM, monitoring, backup, replication, and newer categories such as private and public clouds. No one conducts hands-on tests of as many IT enterprise products as we do.

We decide which product categories to test based on current trends and on what our IT customers are interested in buying. Our experienced product specialists consider market data to help them select specific products, with a focus on representative price ranges, and important features. They also contact manufacturers to make sure products will still be available when our Evaluations are published.

As with all test programs at MagLar, buys virtually all products anonymously in the channel, or direct from the vendor like any enterprise IT customer. That enables us to remain completely impartial and unbiased.

Our buyers across the US and Europe make their purchases, without revealing that the products are destined for the MagLar labs.

We also provide information on the services integral to the use of IT products, including software maintenance and support. The MagLar regularly surveys enterprise IT to see how satisfied they are with those services overall and in specific areas such as cost and quality.

When we test

For the key categories mentioned above, we produce new Evaluations almost every month, and we update our Evaluations of other types of products several times a year. We also test innovative, high-interest products such as the HP Cloud individually as soon as they are available. When time is of the essence, we occasionally buy a new product directly from the manufacturer before it is released to the channel.

Where we test

We conduct our tests in laboratories with equipment designed for the demands of specific products. For example, private cloud software products are tested in a datacenter with more than a hundred virtualization enabled servers to understand the features, performance, usability, scalability and reliability under load and stress.

How we test

IT products are tested by engineers and technicians with years and sometimes decades of expertise in their field. They live with the products for several weeks, putting them through a battery of objective tests using scientific measurements, along with subjective tests that replicate the user experience. We test products against existing industry standards and develop our own benchmarks when we encounter new technologies or issues that require further testing. All products within a category go through exactly the same tests, side by side, so they're judged on a level playing field, and test results can be compared.

Testers focus on a product's primary function (evaluating dependency mapping capabilities for CMDBs, for example) and some secondary functions, too (scalability for private clouds or restoration time for backup products). We evaluate whether features add to usability or just make the software more complicated. We also check where appropriate, security, reliability, performance, and other attributes.

We use a product as any IT administrator and end user would. For example, we assess how long it takes for a cloud solution to provision a new service for an end user.

What we report

We combine the results of all those tests to arrive at individual scores for key attributes. Those scores, in turn, contribute to an overall score that enables us to rank products. The result: our unique Evaluations, which provide an at-a-glance comparison of tested products. Then we go a step further, recommending the products best-suited to specific needs and requirements, and providing detailed test results on each vendor.