Criteria for evaluating industrial automation software - reports & dashboardsCriteria for Selecting Industrial Reporting and Dashboard Software

Deciding that you need a new software solution to enhance your quality, productivity or increase your competitiveness is the first step.  The second is following a solid evaluation process.  This article will help to highlight the key areas of focus for that evaluation.

The selection of a strategic software solution involves evaluating that solution over 5 key areas:

  • Authoring Company Evaluation
  • Channel Evaluation
  • Vertical Market Fit Evaluation
  • Product Capabilities Evaluation
  • Overall Technology Evaluation

Let’s break these areas down one at a time and highlight why they are important and sub areas to consider in the process.

Authoring Company Evaluation

Review company health, growth and longevity in your specific market.  We’re talking about a strategic purchase, one that will require acquisition, training, installation, configuration, operation and long term maintenance.  While these will be important in other steps of this evaluation, they highlight the considerable investment you will be making and that this investment shouldn’t be taken lightly.  You’ll want to make sure the company you select is in this business for the long haul and that they will be delivering both ongoing support and enhancements for your intended use.

  • Is the company focused in your area of business?
  • Has the company been around for a significant period and are do they appear sustainable?
  • Does the company have a reputation that is beneficial to its future?
  • Does the company have partnerships that will bolster its success?
  • Is the company delivering a solution for a Global Market?
  • Do you have peers that are already using the solution?

Channel Evaluation

Many products are brought to market through partnerships with sales channels and system integration partners (channels).  While these are collaborations with the authoring company, and you can always access a product through a different representative, distributor or system integrator, you’ll still want to evaluate your primary supplier for competency, quality and dependability.

  • Is the software available from a channel you trust and have dealt with before?
  • Are they local (accessible) and do they have the domain expertise you need?
  • Can the channel train you, deliver the solution and support you?
  • Does the channel meet the applicable criteria in the previous (Author) section?
  • Does the channel have a vested interest in your success with this product?

Vertical Market Fit Evaluation

Software is typically a tool to be applied to any application.  Microsoft Word is a general purpose word processor that can create legal (vertical market) documents, but it isn’t built for Lawyers.  Excel is a general purpose spreadsheet tool that can perform production (vertical market) statistics, but it isn’t built specifically for those markets.  Microsoft SQL Server Reporting Services and SAP Crystal Reports are general purpose reporting tools that with the right effort, can create compliance and performance reports and dashboards in vertical markets, but they will have gaps in functionality that will need filling with significant integration efforts.  While general purpose software can be used for vertical market applications, software solutions that are purpose built to fit the needs of a vertical market will deliver significant advantages both in short term configuration and long term total cost of ownership that should not be overlooked.

  • Is there a proven track record in your vertical market?
  • Is the software product widely used in your specific application?
  • Can you clearly identify the unique benefits for your vertical market?
  • Does the supplier have vertical market expertise and support?
  • Will it deliver what is needed out of the box, without additional customization or special integration efforts?

Product Capabilities Evaluation

The application should deliver to your requirements with the least level of effort.  With the breadth of options available to a user, it is tempting to select one application to perform one function and another for another function.  Added integration between the applications can deliver a complete solution.  But over time you’ll find that solutions requiring maintenance of the various pieces, management of interfaces and user customizations, will drive a greater long term cost of ownership and lower overall reliability.

  • Does the solution provide connectivity that you require now and in the future?
  • Is the solution well documented and is there online help with search?
  • Does the solution deliver the statistics you need now and in the future?
  • Is the solution configurable or does it require a degree of specialized knowledge?
  • Are there tools to customize the solution?
  • Can the solution be used by your target audience or does it need IT support?
  • Does the solution offer flexibility and scalability?
  • Is the solution receiving security and feature updates?
  • Does the solution offer notification and information delivery?
  • Does the solution offer a Web Portal and or Cloud capabilities?
  • Is the Web Portal Mobile form-factor aware?
  • Is High Availability an option with this solution?
  • Does the solution offer version management, audit trails and user security?
  • Will the solution support operation on virtual machines?
  • Does the solution address special vertical market needs?
  • How is licensing handled – soft licenses and Dongle options?
  • Does the solution offer productivity enhancement tools?
  • Will the solution integrate with other technologies you use – Excel, RDB, etc.?
  • Is there an integrated security model for small applications and domain based security for enterprise installations?
  • Is localization a core feature of the solution in both development and user modes?

Overall Technology Evaluation

Technology speaks to the architecture of your proposed solution.  Technology is always a moving target and the selection of the most appropriate technologies for the various tasks to be performed by a solution will help to deliver optimal performance, greater scalability, and broader portability and can enable the solution authors to adapt and deliver additional functionality with greater ease over the lifetime of your installation.  Major shifts in technology include Windows applications displacing DOS applications, .NET replacing C, Databases replacing flat files, HTML5 replacing Silverlight, Active-X and JAVA applications.  No one technology is appropriate for an entire product.  Products are typically an integration of relevant technologies best suited to deliver on the need at hand.  The best approach in this area is to have a detailed review of the development methodology of a solution and a review and justifications of the technologies in use and a justification of their benefits.

  • Does the product leverage high performance architectures and development methods for its engine?
  • Does the tool leverage higher level development tools for its UI and development environment?
  • How is a Web Portal handled? What is the underlying Web Server? What does the Web Server deliver as its Web technology?
  • Which operating systems are supported?
  • Has the solution passed an ICS-CERT (https://ics-cert.us-cert.gov/) audit?
  • Is Windows Domain Security supported?
  • Are Industry Standards supported?
  • Is there a development and anomaly methodology in place?
  • How the solution is documented and is there online help?
  • What license technology is in use?
  • How scalable is the solution and what architectures are in place for scalability?

Once you’ve created your criteria list for an evaluation, then it’s on to the comparison.  There are a range of methods and metrics for evaluating products against one another.  The most common technique is through some form of Kepner-Tregoe (KT) analysis.  A Kepner-Tregoe analysis refers to a process of weighing alternatives in which a person lists and assigns a numerical weight to a series of values (noting that some may be absolute requirements), gives each alternative a numerical rating according to each value, and computes a numerical score for each alternative.  A quick search on Google will highlight sources and methods for creating a KT Spreadsheet.

If you have a need in Industrial Reporting and Performance Dashboards, then please consider Dream Report as an alternative to consider.  We’ve created a simple form to assist with your evaluation and would be happy to respond to any technical questions that you may have.

Dream Report vs Alternatives Worksheet