|
How we Evaluate
IT
products
When it comes to testing
enterprise IT products, MagLar stands head and shoulders above the rest, with
in-depth hands on testing of the leading IT solutions . You
might know us best for our Evaluations, which enable you to compare the features
and performance of
many vendors at a glance.
To get the most for your
IT budget,
here is an overview of our unique hands-on testing process.
What we test
MagLar
currently tests about 100 enterprise IT products every year, in seven or so
categories. We cover everything from data protection and CMDBs to
virtualization, ITSM, monitoring, backup, replication, and newer
categories such as private and public clouds. No one conducts hands-on tests of
as many IT enterprise products as we do.
We decide which product categories to
test based on current trends and on what our IT customers are interested in
buying. Our experienced product specialists consider market data to help them
select specific products, with a focus on representative price ranges, and
important features. They also contact manufacturers to make sure products will
still be available when our Evaluations are published.
As with all test programs at MagLar,
buys virtually all products anonymously in the channel, or direct from the
vendor like any enterprise IT customer. That enables us to remain completely
impartial and unbiased.
Our buyers across the US and Europe make their
purchases, without revealing that the products are destined for the MagLar
labs.
We also provide information on the
services integral to the use of IT products, including software
maintenance and support.
The MagLar regularly surveys enterprise IT to see how satisfied they are with
those services overall and in specific areas such as cost and quality.
When we test
For the key categories mentioned above,
we produce new Evaluations almost every month, and we update our Evaluations of
other types of products several times a year. We also test innovative,
high-interest products such as the HP Cloud individually as soon as they are
available. When time is of the essence, we occasionally buy a new product
directly from the manufacturer before it is released to the channel.
Where we test
We conduct our tests in laboratories with
equipment designed for the demands of specific products. For example, private
cloud software products are tested in a datacenter with more than a hundred
virtualization enabled servers to understand the features, performance,
usability, scalability and reliability under load and stress.
How we test
IT products are tested by engineers and
technicians with years and sometimes decades of expertise in their field. They
live with the products for several weeks, putting them through a battery of
objective tests using scientific measurements, along with subjective tests that
replicate the user experience. We test products against existing industry
standards and develop our own benchmarks when we encounter new technologies or
issues that require further testing. All products within a category go through
exactly the same tests, side by side, so they're judged on a level playing
field, and test results can be compared.
Testers focus on a product's primary function
(evaluating dependency mapping capabilities for CMDBs, for example) and some
secondary functions, too (scalability for private clouds or restoration time
for backup products). We evaluate whether features add to usability or just
make the
software more complicated.
We also check where appropriate, security, reliability, performance, and other
attributes.
We use a product as any IT administrator and
end user would. For example, we assess how long it takes for a cloud solution
to provision a new
service for an
end user.
What we report
We combine the results of all those tests
to arrive at individual scores for key attributes. Those scores, in turn,
contribute to an overall score that enables us to rank products.
The
result: our unique Evaluations, which provide an at-a-glance comparison of
tested products.
Then
we go a step further, recommending the products best-suited to specific needs and
requirements, and providing detailed test results on each vendor.
|