Lims Evaluation Essay

The intention of this document is 1. to assist labs searching for a laboratory informatics product with identifying their system needs and 2. to help labs better determine if a specific vendor/product meets their requirements in the form of a request for information (RFI). The idea is to allow users to incorporate a standardized specifications sheet in their comparison of various LIMS and other laboratory informatics products.

This questionnaire lists the extensive requirements of a LIMS as well as other laboratory informatics systems. (See below for more about this.) The questionnaire is organized such that sections 1.0 through 1.4 offer questions applicable to most any laboratory informatics system, be it a LIMS or an ELN. Section 1.5 covers functionality found specifically in software systems other than LIMS.

This questionnaire is comprehensive and includes many items that do not apply to every lab. Additionally, some laboratories' requirements include a functionality item not common to other labs. Section 1.6 "Industry-specific" contains a selection of those industry-specific requirements and will continue to be amended over time. The last section, 1.7 "Custom functions," is designed for the vendor to insert any additional functionality that doesn't fall under the categories provided.

When referencing a particular item for someone else, use the section number followed by the requirement letter, e.g. 1.4.2.f for "Does your system allow the administrator to create custom screens, applications, and reports? Please give details."

More about this questionnaire

As noted above, this questionnaire was originally designed to cover aspects of a laboratory information management system (LIMS). However, a significant portion of this page if not most of it could easily apply to other laboratory informatics systems like ELNs. As such, we took the approach of adding addenda (as seen in section 1.5 and 1.6) that provide additional requirements unique to other systems and industries. If you're evaluating several industry-neutral LIMS, you likely don't need 1.5 and 1.6.

Requirement code and notes

In responding to each requirement, the vendor must select a requirement code from the following:

  • Y: Meets requirement in commercial off-the-shelf solution as delivered/configured (or vendor provides service)
  • YC: Meets requirement only with customization (additional code, using a third-party application, etc.)
  • N: Does not meet requirement
  • I: Informational response only, N/A

The vendor should ideally enter a requirement code and a response for each functionality question.

Printing or saving

The print/export options in the navigation on the far left give you a few options for saving this and other pages, printing them for later.

Notice you'll have to open 1.5 and 1.6 in separate pages to view, print, and save them.

Note: Tables are currently not being rendered in PDFs as we like through the MediaWiki Collection extension. While we sure would love for you to make a PDF of this document, be warned: the tables don't format as intended, using either the "Create a book" or "Download as PDF" processes. For now you're best off selecting "Printable version" and printing that instead.

1.0 Vendor information

1.0 Vendor information
Company name
Physical address
LIMSwiki web page
Contact name and title
Contact e-mail
Contact phone and fax
Years in business

1.1 Vendor services

1.1 Vendor services
Request for information Requirement code Vendor response
a. The vendor offers an online and/or on-site demonstration.
b. The vendor provides a detailed project plan that includes the project team, timeline, deliverables, and risk and issue management procedures.
c. The vendor explains their overall project approach, deliverables, time constraints, and any other criteria for the project.
d. The vendor provides reliable cost estimates and pricing schedules, including all products and services in the scope of work.
e. The vendor details the amount of time and staff that the purchaser will have to provide for the implementation process.
f. The vendor can explain the maintenance and support offered during and after implementation, including times and methods of availability, issue escalation and management, etc.
g. The vendor provides a support schedule for the implementation process, including optional support levels, their function, and availability.
h. The vendor provides support during the "go-live" period between system validation/operational deployment and final acceptance/beginning of maintenance and support agreements.
i. The vendor provides a gap analysis after initial system installation, identifying the deliverables or tasks remaining.
j. The vendor provides a table linking each deliverable to the corresponding user requirement specification it fulfills.
k. The vendor uses a consistent training methodology for educating new users.
l. The vendor supplies LIS-specific training program curricula.
m. The vendor provides user, administrator, developer, installation, and reference manuals.
n. The vendor provides design qualification documentation.
o. The vendor provides installation qualification documentation.
p. The vendor provides operation qualification documentation.
q. The vendor provides performance qualification documentation during implementation.
r. The vendor provides well-documented system upgrades that authorized users can independently install.
s. The vendor provides source code for the system.
t. The vendor provides an optional comprehensive set of test codes suitable for use by the purchaser.

1.2 Information technology

1.2.1 General IT

1.2.1 General IT
Request for information Requirement code Vendor response
a. The system operates with a traditional client-server architecture, with software installed on each machine that needs to access the system.
b. The system operates with a web-based interface, hosted on a server and accessed via a web browser on most any machine.
c. The system contains a single, centralized database that supports multiple sites and departments.
d. The system's database conforms to the Open Database Connectivity Standard (ODBC).
e. The system is designed so upgrades to the back-end database do not require extensive reconfiguration or effectively cripple the system.
f. The system is designed to not be impacted by multiple users or failover processes.
g. The system applies security features to all system files.
h. The system applies log-in security to all servers and workstations accessing it.
i. The system provides a workstation and server authentication mechanism.
j. The system applies Secured Socket Layer (SSL) encryption on the web client interface.
k. The system encrypts client passwords in a database, with support for multi-case and special characters.
l. The system uses TCP/IP as its network transport.
m. The system provides data archiving functionality, including a configurable scheduled archive, for all contained data, without requiring an off-line mode or human interaction with the data to be archived.
n. The system allows automated backup and restore capability without support intervention, as well as manual backups.
o. The system maintains the transactional history of system administrators.
p. The system maintains an analyst communication log, accessible by the administrator.

1.2.2 Hardware environment

1.2.2 Hardware environment
Request for information Requirement code Vendor response
a. The system proves compatible with a variety of hardware environments.
b. The system can be utilized with a touch-screen.

1.2.3 Software environment

1.2.3 Software environment
Request for information Requirement code Vendor response
a. The system proves compatible with a variety of software environments.
b. The system utilizes a non-proprietary database such as Oracle or Microsoft SQL Server.

1.3 Regulatory compliance and security

1.3.1 Regulatory compliance

1.3.1 Regulatory compliance
Request for information Requirement code Vendor response
a. The system supports 21 CFR Part 11 and 40 CFR Part 3 requirements, including login security, settable automatic logouts, periodic requirements for mandatory password changes, limits on reusability of passwords, and full electronic signature.
b. The system supports ISO/IEC 17025 requirements.
c. The system supports HIPAA requirements.
d. The system supports GALP and/or GAMP standards.
e. The system supports the standards of The NELAC Institute.
f. The system meets government requirements for handling classified information and documents.
g. The system maintains audit and specification violation trails of all data manipulation — such as result and header information changes — as consistent with all applicable regulations and standards.
h. The system maintains audit trails at least as long as the records to which they pertain.
i. The system's audit log retains all data, prohibits any deletions, allows user comments, and allows reporting of contained information.
j. The system provides additional persistent auditing capabilities, such as the audit of cancelled tests and scheduled system functions.
k. The system provides user-selectable NELAP-compliant internal chain of custody that tracks all specimens and associated containers from the time they are collected until disposed of.
l. The system provides the ability to insert/manage secure electronic and/or digital signatures.
m. The system incorporates automatic date and time stamping of additions, changes, etc.
n. The system can automatically validate and approve data prior to being moved to the main database.

1.3.2 Security

1.3.2 Security
Request for information Requirement code Vendor response
a. The system allows administrators and other authorized users to configure multiple levels of user rights and security by site location, department, group, role, and/or specific function.
b. The system allows administrators and users to reset user passwords.
c. The system features and enforces adjustable rules concerning password complexity, reuse, and expiration.
d. The system can lock a user out after a specified number of consecutive failed log-in attempts.
d. Does your system provide automatic logout based on keyboard or mouse inactivity?
e. The system provides the option for automatic user logout based on keyboard or mouse inactivity.
f. The system makes authority checks to ensure only authorized individuals can use the system to perform an operation.
g. The system allows authorized users to modify records, while also maintaining an audit trail of such actions.
h. The system allows authorized users to manually delete records, while also maintaining an audit trail of such actions.
i. The system prompts users to declare a reason for making changes to or deleting data in the system.
j. The system provides email notification of lockout, security access, and improper workstation access.
k. The system provides a mechanism to allow a user read-only access to stored data.
l. The system allows authorized users to generate a detailed user access record.
m. The system allows authorized users to review audit logs at will.
n. The system allows authorized users to query and print chain of custody for items, cases, projects, and batches.
o. The system allows supervisors to override chain of custody.
p. The system automatically tracks when supervisors review critical result values.
q. The system allows automatic and/or manual holds or locks to be placed on data to ensure it goes unaltered or remains retrievable during a retention period.
r. The system can first feed data from connected non-CFR-compliant instruments through a virtual environment that is compliant (audit trailed, secure, versioned, etc.) before being stored.
s. The system can control whether users are able to export data to portable long-term storage media like a USB flash drive or recordable DVD.
t. The system employs automatic file encryption on stored data.
u. The system employs checks to enforce permitted sequencing of steps and events.
v. The system allows multiple users to connect simultaneously to a contract lab.
w. The system provides read-only access to contract laboratory results.
x. The system prohibits issuing reports outside of qualified areas while also allowing reports to be viewed locally or remotely based on security application limits and/or specimen ownership.

1.4 General system functions

1.4.1 General functions

1.4.1 General functions
Request for information Requirement code Vendor response
a. The system offers non-LIMS trained personnel the ability to easily access system data via an intuitive, user-friendly Windows-type graphical user interface (GUI) which permits the display of data from specimen points, projects, and user-defined queries, and can be configured to language, character set, and time zone needs.
b. The system allows authorized users to configure their GUI to a specific language, character set, and time zone.
c. The system permits remote access for users, system admins, and support agents.
d. The system allows for the use of navigation keys to freely move from field to field.
e. The system allows tabular data to be sorted and filtered.
f. The system can send on-screen output to a printer or file without contradicting view-only statuses.
g. The system provides single data entry, automatically populates other data fields, and remembers pertinent and relevant data so it doesn't need to be re-entered, selected, or searched for.
h. The system eliminates (or significantly reduces) redundant data entry and paper trails.
i. The system contains one or more spell-check dictionaries that allow authorized users to add, edit, or remove entries.
j. The system provides full database keyword and field search capability, including the use of multiple search criteria.
k. The system includes the ability to search multiple databases, including those containing legacy data.
l. The system allows users to build, save, and edit queries for future use.
m. The system can automate the search for and extraction of pertinent data, including the export of that data to external applications for additional processing and calculation.
n. The system cleanly converts migrated data to allow for reporting of historical specimen collections.
o. The system allows for the specification of a retention period for captured data and can enact it based on date-based fields or a future event.
p. The system can manage and store both sample- and non-sample-related data, including images from microscopes, GCMS scans of peaks, PDF files, spreadsheets, or even raw data files from instrument runs for later processing.
q. The system can manage and store media objects like digital photos, bitmaps, movies, and audio files.
r. The system issues sequential numbers for chain of custody.
s. The system's numbering scheme allows for sub-numbering while maintaining parent-child relationships.
t. The system efficiently utilizes standardized data input points and enhanced individual workload tracking.
u. The system captures data from all laboratory processes, ensuring uniformity of statistical reporting and other electronic data shared with designated users of the data.
v. The system can link objects to other objects, e.g. linking a standard operating procedure (SOP) to a test result.
w. The system notifies users of events like the scheduling, receipt, and completion of tasks.
x. The system includes the ability to set up alerts via email.
y. The system has real-time messaging capabilities, including instant messaging to one or more users.
z. The system supports the use of a voice recognition system (for navigation or transcription) or has that functionality.
aa. The system offers integrated or online user help screens.

1.4.2 Configuration and customization

1.4.2 Configuration and customization
Request for information Requirement code Vendor response
a. The system can be configured to meet the workflow of a laboratory without additional programming.
b. The system architecture is modular or extensible and can easily and efficiently be modified to facilitate the addition of new functionality as business needs change.
c. The system has an application programming interface (API) or a similar software development toolkit (SDK). If web-based, the API should support Simple Object Access Protocol (SOAP), representational state transfer (REST), or both.
d. The system can expand to accommodate a new discipline.
e. The system supports customized screens with user-definable information specific to a customer, department, analysis, etc.
f. The system allows the administrator to create custom screens, applications, and reports.
g. The system allows a user to independently add fields without requiring reconfiguration of the system, even after routine upgrades and maintenance.
h. The system allows a user to independently add universal fields on all samples logged into the system at any time during or after implementation, while neither voiding the warranty nor requiring vendor review at a later date.
i. The system supports the definition and maintenance of edit tables and lists.
j. The system dynamically changes captions (labels) on system fields.
k. The system has dynamically configurable limit periods and notification hierarchy.
l. The system allows for the integration of additional printers and scanners both locally and externally.

1.4.3 Receiving and scheduling

1.4.3 Receiving and scheduling
Request for information Requirement code Vendor response
a. The system tracks status and workflow of the accession throughout the laboratory lifecycle, from submission to final analysis, including receiving, diagnostic testing, diagnostic test result reporting, and billing.
b. The system supports barcoded specimen labeling and tracking.
c. The system allows users to create, manage, and track viewable sample container schemata.
d. The system creates and maintains a unique electronic accession record for each accession received.
e. The system supports standard-format digital picture and document upload and attachment to electronic accession records.
f. The system supports a user-configurable, spreadsheet-style, templated multi-specimen (batch) login without requiring additional programming.
g. The system supports the modification of specimen or specimen batch information prior to actual multi-sample (batch) login.
h. The system supports ad-hoc specimens not predefined in the specimen point list during multi-sample (batch) login.
i. The system creates, saves, and recalls pre-login groups for routine samples to simplify recurring logins.
j. The system streamlines the login of recurring sample projects.
k. The system automatically generates labels for recurring samples and sample groups.
l. The system allows authorized users to generate user-definable or rules-based chain of custodies, worksheets, routing sheets, and custom labels upon sample login.
m. The system provides a comprehensive view of all samples and projects in the system using a color-coded status view of the current and scheduled samples via user-configurable templates, all without requiring additional programming.
n. The system includes environmental monitoring (EM) functionality or integrates with an external EM product.
o. The system prevents a sample from being placed in a report queue until approved.
p. The system includes comprehensive scheduling, tracking, and flow management of samples, including across multiple sites.
q. The system is capable of geographically organizing sample records and their associated tests for special analysis and reporting.
r. The system allows authorized users to accept, cancel, re-run, and override attributes of one or multiple tests for a given sample.
s. The system allows authorized users to review the available test types in the system, including their reference range and units of measure.
t. The system has a "miscellaneous" test code to allow a test undefined in the system to be ordered and billed.
u. The system allows authorized users to schedule routine samples on an hourly, daily, weekly, or monthly basis, allowing them to be enabled and disabled as a group.
v. The system generates an hourly, daily, weekly, or monthly sample collection schedule from a scheduling database.
w. The system schedules and assign tasks based on available inventory and personnel.
x. The system supports automatic assignment and scheduling of analysis requests.
y. The system receives accession/analysis request information from web-enabled forms.
z. The system electronically receives and processes collection and analysis request information and schedules from third parties.
aa. The system has an inter-lab transfer function.
ab. The system processes automated uploading of field-derived sample collection data.
ac. The system allows users to handle billable and non-billable tests on the same accession.
ad. The system supports tracking of shipping and receiving.

1.4.4 Analysis and data entry

1.4.4 Analysis and data entry
Request for information Requirement code Vendor response
a. The system supports a variety of test protocols, each capable of storing test comments, test required, and special information like GCMS conditions or special objects associated with the test.
b. The system provides and enforces normal data range values for diagnostic tests.
c. The system includes default input values for diagnostic tests.
d. The system provides for a single test code requiring multiple analytes as targets.
e. The system allows authorized users to make a test code inactive across one or more sites while retaining it for audit and reporting purposes.
f. The system limits test code authorization to only qualified personnel and maintains their certification(s) to run assigned tests.
g. The system supports and qualifies text-based tests.
h. The system supports single-component tests such as pH, BOD, CD, etc.
i. The system allows users to specify a single-component, multi-component, or narrative text test or group of tests, which represent all tests required.
j. The system can effectively manage complex molecular testing protocols.
k. The system can effectively manage genetic testing protocols.
l. The system permits user-generated and modifiable calculations (based on a formulaic language) to be applied to all tests.
m. The system distinguishes between routine and duplicate analysis.
n. The system provides an overview of all outstanding tests/analyses for better coordination of work schedules.
o. The system notifies analysts of applicable safety hazards associated with a sample, reagent, or test before testing begins.
p. The system electronically transfers an item during testing from one functional area to another.
q. The system's user interface displays visual indicators such as status icons to indicate a sample's status in the workflow.
r. The system allows file transfer of data from instruments via intelligent interfaces or multi-specimen/multi-test ASCII files, with full on-screen review prior to database commitment.
s. The system permits manual data entry into an electronic worksheet of test measurements and results.
t. The system allows incorrectly inputted data to be manually corrected.
u. The system prevents out-of-range and other critical results from being posted as final.
v. The system provides colored visual indication of previously entered data as well as new data associated with a single sample when a result is entered, with the indicator changing color if the value is out of specification.
w. The system allows automated or semi-automated data insertion.
x. The system stores non-narrative textual results in searchable fields.

1.4.5 Post-analysis and validation

1.4.5 Post-analysis and validation
Request for information Requirement code Vendor response
a. The system updates sample/item status when tests are completed.
b. The system automatically reorders a test or orders additional tests if results don't meet lab-defined criteria, especially when the original sample is still available.
c. The system reads results from previously entered tests to calculate a final result and immediately displays the calculated result.
d. The system allows authorized users to review all analytical results, including pricing, spec violations, history or trend analysis by analyte, and comments.
e. The system graphically displays the results of one or more tests in a graph (normalized or otherwise) for the purpose of visualizing data or searching for possible trends.
f. The system allows on-screen review of the stored test result, diluted result with corrected method detection limits (MDLs), and qualifiers after running samples for multiple dilutions as in gas chromatography–mass spectrometry (GC-MS).
g. The system includes data mining tools for model learning, evaluation, and usage.
h. The system displays the standard operating procedure (SOP) associated with each test result to ensure proper techniques were used.
i. The system stores test-related analysis comments with the test.
j. The system provides auto-commenting for common laboratory result comments.
k. The system is capable of displaying entered order and test comments as an onscreen alert for testing and other personnel.
l. The system provides for high-volume multi-component transfers of test results, with the ability to automatically match samples to data files in either a backlog mode or a designated file mode, to parse the data, and to review and commit the sample data.
m. The system's results validation process accesses all information about a sample or group of samples, including comments or special information about the sample.
n. The system's results validation process checks each result against its individual sample location specifications (both warning and specification limits).
o. The system supports validation at the analysis and sample level, while also prohibiting sample validation when analysis validation is incomplete.
p. The system uses a menu-driven process for results validation.
q. The system provides secure electronic peer review of results.
r. The system clearly differentiates released preliminary data from fully validated results.
s. The system validates/approves data prior to being moved to the main database.
t. The system can hold all test results on a sample with multiple tests ordered on it until all work is completed and a final report is issued.
u. The system fully manages all aspects of laboratory quality control, including the reporting and charting of all quality control data captured in the lab.
v. The system provides a base for a quality assurance program, including proficiency testing, scheduled maintenance of equipment, etc.
w. The system distinguishes QA/QC duplicates from normal samples.
x. The system allows QA/QC tests to be easily created and associated with the primary analytical test.
y. The system allows manual entry of QA and QC data not captured as part of the system's regular processes.
z. The system calculates monthly QA/QC percentages for testing.
aa. The system automatically flags out-of-range quality control limits.
ab. The system is able to flag results for aspects other than being out-of-range, including testing location, test subject age, etc.
ac. The system checks data files for specification and corrects them for specific reporting and analyte limits and qualifiers like dilution factor, automatically assigning qualifiers based on project analyte limiting.

1.4.6 Instruments

1.4.6 Instruments
Request for information Requirement code Vendor response
a. The system bilaterally interfaces with instruments and related software.
b. The system downloads data directly from laboratory instruments.
c. The system permits the defining and exporting of sequences to instruments.
d. The system tracks and reports on the usage of attached laboratory instruments.
e. The system allows automatic or manual reservation/scheduling of laboratory instruments.
f. The system automatically (or manually allow an authorized user to) removes an instrument from potential use when it falls out of tolerance limit or requires scheduled calibration.
g. The system provides a database of preventative maintenance, calibration, and repair records for laboratory equipment, preferably supported by standardized reporting.
h. The system schedules calibration, verification, and maintenance tasks in the worksheets or work flow process and make that schedule available for viewing.
i. The system allows users to create and edit instrument maintenance profiles.

1.4.7 External system interfaces

1.4.7 External system interfaces
Request for information Requirement code Vendor response
a. The system supports a library of common electronic data deliverable (EDD) formats.
b. The system transfers data to and from another record management system.
c. The system integrates with Microsoft Exchange services.
d. The system imports data from and exports data to Microsoft Word, Excel, and/or Access.
e. The system can interface with non-Microsoft programs.
f. The system interfaces with external billing systems.
g. The system interfaces with enterprise resource planning (ERP) systems.
h. The system interfaces with external contract or reference laboratories to electronically send or retrieve datasheets, analysis reports, and other related information.
i. The system exchanges data with National Identification System (NAIS) tracking systems.
j. The system generates and exchanges data with other systems using Health Level 7 (HL7) standards.
k. The system leverages the application programming interface (API) of other systems to establish integration between systems.
l. The system provides a real-time interface for viewing live and stored data transactions and errors generated by interfaced instruments and systems.
m. The system transmits status changes of specimens, inventory, equipment, etc. to an external system.
n. The system directs output from ad-hoc queries to a computer file for subsequent analysis by other software.
o. The system supports the manual retransmission of data to interfaced systems.
p. The system supports dockable mobile devices and handle information exchange between them and the system.
q. The system supports the use of optical character recognition (OCR) software.

1.4.8 Reporting

1.4.8 Reporting
Request for information Requirement code Vendor response
a. The system includes a versatile report writer and forms generator that can generate reports from any data in tables.
b. The system includes a custom graphic generator for forms.
c. The system interfaces with a third-party reporting application.
d. The system allows the development of custom templates for different types of reports.
e. The system maintains template versions and renditions, allowing management and tracking of the template over time.
f. The system generates template letters for semi-annual reports.
g. The system supports report queries by fields/keys, status, completion, or other variables.
h. The system use Microsoft Office tools for formatting reports.
i. The system supports multiple web browsers for viewing online reports.
j. The system generates, stores, reproduces, and displays laboratory, statistical, and inventory reports on demand, including narrative.
k. The system includes several standard reports and query routines to access all specimens with the pending status through a backlog report that includes the following criteria: all laboratory, department, analysis, submission date, collection date, prep test complete, location, project, specimen delivery group, and other user-selectable options.
l. The system indicates whether a report is preliminary, amended, corrected, or final while retaining revision history.
m. The system supports both structured and synoptic reporting.
n. The system generates management and turn-around time reports and graphs.
o. The system generates customized final reports.
p. The system automatically generates laboratory reports of findings and other written documents.
q. The system automatically generates individual and aggregate workload and productivity reports on all operational and administrative activities.
r. The system automatically generates and transmits exception trails and exception reports for all entered and/or stored out-of-specification data.
s. The system generates a read-only progress report that allows for printed reports of specimen status and data collected to date.
t. The system provides an ad-hoc web reporting interface to report on user-selected criteria.
u. The system automatically generates and updates control charts.
v. The system generates QA/QC charts for all recovery, precision, and lab control specimens via a full statistics package, including Levy-Jennings plots and Westgard multi-rule.
w. The system displays history of previous results for an analyte's specimen point in a tabular report, graphic trend chart, and statistical summary.
x. The system automatically generates and posts periodic static summary reports on an internal web server.
y. The system transmits results in a variety of ways including fax, e-mail, print, and website in formats like RTF, PDF, HTML, XML, DOC, XLS, and TXT.
z. The system electronically transmits results via final report only when all case reviews have been completed by the case coordinator.
aa. The system includes a rules engine to determine the recipients of reports and other documents based on definable parameters.
ab. The system allows database access using user-friendly report writing and inquiry tools.

1.4.9 Laboratory management

1.4.9 Laboratory management
Request for information Requirement code Vendor response
a. The system allows the creation, modification, and duplication of user profiles.
b. The system allows entry, maintenance, and administration of customers, suppliers, and other outside entities.
c. The system allows customers, suppliers, and other such entities to be flagged as either active or inactive.
d. The system allows the creation, modification, and maintenance of user training records and associated training materials.
e. The system supports the ability to set up separate security, inventory, reporting, etc. profiles across multiple facilities.
f. The system allows the management of information workflow, including notifications for requests and exigencies.
g. The system allows the management of documents like SOPs, MSDS, etc. to better ensure they are current and traceable.
h. The system allows the management and monitoring of resources by analyst, priority, analysis, and instrument.
i. The system allows authorized persons to select and assign tasks by analysts, work group, instrument, test, sample, and priority.
j. The system allows authorized persons to review unassigned work by discipline and by lab.
k. The system allows authorized persons to review pending work by analyst prior to assigning additional work.
l. The system manages and reports on reference samples, reagents, and other inventory, including by department.
m. The system automatically warns specified users when inventory counts reach a definable threshold and either prompt for or process a reorder.
n. The system allows authorized users to monitor and report on reference and reagent creation, use, and expiration.
o. The system allows authorized users to search invoice information by invoice number, account number, accession, payment types, client, or requested diagnostic test(s).
p. The system includes performance assessment tracking.
q. The system receives, records, and maintains customer and employee feedback and applies tools to track the investigation, resolution, and success of any necessary corrective action.
r. The system includes an incident tracking system for recording, investigating, and managing safety and accident violations in the laboratory.
s. The system monitors proficiency test assignment, completion, and casework qualification for analytical staff.
t. The system includes revenue management functionality, including profitability analysis.
u. The system provides analysis tools to better support laboratory functions like resource planning, productivity projections, workload distribution, and work scheduling, and those tools display information in a consolidated view, with the ability to drill down to more detailed data.
v. The system calculates administrative and lab costs.
w. The system captures and maintains patient, submitter, supplier, and other client demographics and billing information for costing, invoicing, collecting, reporting, and other billing activities.
x. The system supports multiple customer payment sources (e.g. grants).
y. The system tracks number of visits per specific industry.

1.5 System-specific

The system-specific addendum can be found here.

1.6 Industry-specific

The industry-specific addendum can be found here.

1.7 Custom requirements

1.7 Custom requirements
Request for information Requirement code Vendor response

The evaluation of hospital laboratory information management systems based on the standards of the American National Standard Institute

Sakineh Saghaeiannejad Isfahani,Reza Khajouei,1Maryan Jahanbakhsh,2 and Mahboubeh Mirmohamadi2

Health Management and Economics Research Center, Isfahan University of Medical Sciences, Isfahan, Iran

1Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, School of Management and Health Information Sciences, Kerman University of Medical Sciences, Kerman, Iran

2School of Medical Management and Informatics, Isfahan University of Medical Sciences, Isfahan, Iran

Address for correspondence: Prof. Reza Khajouei, Medical Informatics Research Centre, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, PO Box: 7618868368, E-mail: moc.oohay@ieuojahk.r

Author information ►Copyright and License information ►

Copyright : © 2014 Isfahani SS.

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.



Nowadays, modern laboratories are faced with a huge volume of information. One of the goals of the Laboratory Information Management System (LIMS) is to assist in the management of the information generated in the laboratory. This study intends to evaluate the LIMS based on the standards of the American National Standard Institute (ANSI).

Materials and Methods:

This research is a descriptive–analytical study, which had been conducted in 2011, on the LIMSs in use, in the teaching and private hospitals in Isfahan. The data collecting instrument was a checklist, which was made by evaluating three groups of information components namely: ‘System capabilities’, ‘work list functions,’ and ‘reporting’ based on LIS8-A. Data were analyzed using the SPSS 20. Data were analyzed using (relative) frequency, percentage. To compare the data the following statistical tests were used: Leven test, t-test, and Analysis of Variance (ANOVA).


The results of the study indicated that the LIMS had a low conformity (30%) with LIS8-A (P = 0.001), with no difference between teaching and private hospitals (P = 0.806). The ANOVA revealed that in terms of conformity with the LIS8-A standard, there was a significant difference between the systems produced by different vendors (P = 0.023). According to the results, a Kowsar system with more than %57 conformity in the three groups of information components had a better conformity to the standard, compared to the other systems.


This study indicated that none of the LIMSs had a good conformity to the standard. It seems that system providers did not pay sufficient attention to many of the information components required by the standards when designing and developing their systems. It was suggested that standards from certified organizations and institutions be followed in the design and development process of health information systems.

Keywords: Evaluation, Hospital information systems, laboratory information management systems, standard


Nowadays, modern laboratories have to operate in a context wherein there is a vast volume of data. The advent of new technologies has led to an increasing growth in both the quality and quantity of information. Such growth may entail some challenges, which make adopting appropriate management methods inevitable.[1] Information management, as a systematic and cost-effective process, can play an efficient role in maintaining, utilizing, and disseminating data.

Collecting sufficient data for identifying the patients and their related samples is regarded as central to the laboratory performance. The collected data must be clear, accessible, complete, timely, and relevant, so that the healthcare professionals can use them to provide quality service.[2,3] A laboratory information management system (LIMS) can be especially helpful in integrating the laboratory with its working operations in such a way that it finally results in the acceleration of all involved processes and sub-processes.[1]

The Laboratory Information Management System is a computer application that is used for storing and managing data provided by healthcare professionals during their daily healthcare routines. This system is also applicable for controlling and managing the samples, reporting test results, and automation of all laboratory processes. It also enables Laboratory Managers to manage their resources, including laboratory staff and tools. Integrating LIMS with information systems of a healthcare facility allows and accelerates the mutual exchange of healthcare date between these two systems. LIMS mainly intends to facilitate the management of a vast amount of information available in the laboratory. This system is very advantageous for Quality Control and Quality Assurance Programs, as it utilizes quality management tools for process monitoring and optimization.[4,5]

On the basis of research findings, accessibility, reliability, and validity of data have a significant effect on the performance of a health information system and contribute to the quality of healthcare services.[6] According to the World Health Organization (WHO), the goal of healthcare information systems is to develop mechanisms to promote the efficient retrieval of patient information to be used for patient care, statistics, and for educational and research purposes.[7] Accessibility of standard data in an organized format is necessary for providing proper and timely medical services by the healthcare providers.[8] Many studies on the use of data in manual or automated formats, emphasize on the importance of appropriate information content for satisfying the healthcare goals.[9,10,11] On the other hand, the type of information presented by the system to the users has a substantial effect on the user-system interaction and subsequently on the efficiency and effectiveness of the provided services. An effective way to enhance the information content of healthcare information systems is standardization of the presented information.[12] A comprehensive set of standards with respect to laboratory information systems has been developed by the ANSI.[13] This study intends to evaluate the information content of hospital LIMSs in teaching and private hospitals, in Isfahan, based on the standards of ANSI. In this study we have specifically used the following standard: Standard Guide for Functional Requirements of Clinical Laboratory Information Management Systems (LIS8-A),

Divergence of LIMSs from the standards can be considered as a main drawback for these systems, which can be opposed to the mission of these systems, that is, increasing accuracy of information, efficiency, effectiveness of work processes, and finally improving patient safety. Evaluation of these systems based on the established standards can shed light on the weaknesses of LIMSs, which are used in many hospitals and healthcare centers. Identification and detection of their problems is the primary step for optimization of these systems.


This applied research is a descriptive–analytical study that was conducted cross- sectionally from June to May 2011. The research population consisted of all LIMSs in use in the 18 teaching and private hospitals in Isfahan (13 teaching and five private hospitals). The LIMSs used by these hospitals were provided by six vendors (Saya_Raya_Ekbatan-e-Hamedan, Lohan, Pouya Samane-ye Diva, Rayavarane Toseah, Kowsar, University Statistical Administration of Isfahan University). Each system was used by one to six hospitals. On account of the small size of the research population, sampling was not performed and the whole research population was studied. Evaluation of the laboratory information system was done using a researcher-designed checklist based on one of the standards of the ANSI and Clinical Laboratory Information Standardization Institute, that is, the LIS8-A standard. LIS8-A covered all the information components related to the performance of the system regarding storing, maintaining, and processing the information, for managerial decision-making.

This study intends to evaluate the functionalities of the LIMSs by reviewing the information components of the systems and their accordance to LIS8-A. These components were classified into three groups: ‘System capabilities’, ‘work list functions,’ and ‘reporting ’ and were reported in terms of both the hospitals and the software provider companies (vendors). The checklist included 63 items. When designing the checklist, the researcher excluded some items of the LIS8-A standard which based on the study objective, were not applicable to the health information systems in Iran. Some extra applicable items were added to the checklist. The content validity of the checklist was affirmed by six experts: A Healthcare Information Management Specialist, a Medical Informatics Specialist, two Software Engineers, a Laboratory Specialist, and a Pathologist. To collect the data, the researcher observed the LIMSs in use in the hospitals and interviewed the users. To analyze the data, if an item was in accordance with the standard it was given score of 1, and if not, it was given a score of 0. Irrelevant items were given no scores. Data were analyzed in SSPS 20 using descriptive–analytical statistics, including, frequency, relative frequency, and percentage. The Leven Test was used to compare the variances and two parametric tests, the t-test and analysis of variance (ANOVA) were used to compare the means. Statistical analysis was performed by using a significance level of α =0.05.


The results of this study on the LIMSs in use in the hospitals showed that the overall mean of accordance with the standard in the three categories was %30.33. According to one-sampled T test, the extent of accordance with the standard of the LIMSs was low (t = 8.91, P = 0.001).

Table 1 presents the extent of LIMS accordance with the LIS8-A standard in the hospitals. There was no significant difference in terms of the accordance of their LIMSs to the standard (t = 0.25, P = 0.806).

Table 1

The extent of conformity of the information content of the laboratory information system used in the selected hospitals with LIS-8 standard

Table 2 shows the extent to which the information content of LIMSs in hospitals conforms to LIS8-A. Based on these results, the conformity of LIMSs in teaching hospitals, in terms of capabilities of the system and work list functions, is better than in private hospitals.

Table 2

The extent of information content conformity to LIS8-A in the hospitals’ laboratory management information systems

The mean scores of the LIMSs conformity to the LIS8-A standard are represented in Table 3, based on the system vendors. The ANOVA statistical analysis revealed a significant difference, in terms of conformity to the standard among the systems provided by different vendors (P = 0.023). Based on this study, the extent of conformity of the LIMSs in five hospitals, which used the Kowsar System, was higher, and in those that used the Pouya Samane-ye Diva, it was lower than that in other hospitals

Table 3

The mean scores for conformity of the information content of the laboratory information management system with the LIS8-A standard based on their vendors

Based on the groups of information components [Table 4], the results of the descriptive statistics showed that the Kowsar system with 72% for ‘system capabilities’ and ‘reporting’ and with 57% for ‘work list functions,’ had a better conformity with the LIS8-A standard than the other systems. The conformity to the standard – except for the ‘reporting’ group in the Lohan System (51%) – was lower than 50% for different groups of information components in other systems.

Table 4

The conformity of different groups of information components to LIS-A standard based on the system vendors


The results of this study revealed that, in total, the extent of conformity of the studied LIMSs to the standard was low, with no significant difference between the systems in teaching and private hospitals. However, conformity to the standard was significantly different based on the vendors of the systems. Low scores of hospital information systems when compared to the standards indicated their poor quality. Based on ISO/IEC9126, the quality of the system in the user's perspective depends on its functionality, reliability, efficiency, effectiveness, as well as its capability for storage and exchange of the data. This standard has been driven from the ISO8402 Standard.[14]

Farzandipour et al.,[15] in their study entitled, ‘Hospital information systems user needs analysis: A vendor surveys,’ compared five major suppliers of hospital information systems in eight teaching hospitals. They found that the vendors of laboratory and pharmacy information systems considered approximately 40% of the users’ needs.

In our study the highest conformity with the standard concerning ‘system capabilities’ was observed in the teaching hospitals with 51% and among vendors in the Kowsar system with 72%, which are still far from the standard. Usability and efficiency are two significant attributes of the system influencing the physicians’ motivation toward using information systems.[16] Both these attributes are directly related to the presentation of the information component on the systems interfaces. According to the results of many studies,[17,18,19] insufficient information components and poor presentation of these components can lead to poor usability, which in turn can result in a higher number of errors made by users, In another study, Ahmadi and Habibi koolaee[20] studied the viewpoints of the nursing staff, clinical users, and the department secretaries concerning functionality of the hospital information systems. This study showed that compared to the clinical users, the nursing staff considered the systems more useful and functional. Paying low attention to or ignoring standard functionalities of an information system in the design process can lower user motivation to use the system and have a negative effect on their adoption and acceptance.[16,20]

This study indicated that none of the LIMSs had a good conformity to the standard. Azizi et al.,[21] in a study, evaluated to what extent the criteria developed by the American College of Physicians (ACP), with regard to the hospital information systems, were followed by the hospitals affiliated with the Iran, Tehran, and Shahid Beheshti Universities. In this study they reported that the conformity of the laboratory information systems in these hospitals with the standards of ACP was medium. The conformity of hospitals in the three universities with the ACP criteria was %29.8, %37.2, and %38.3, respectively. These scores reflect a low conformity of the existing systems with the standards. These results are consistent with the results of our study.

Besides our study, a research has so far been conducted on the compliance of the Iranian Health Information Systems, with users requirements or the existing standards, and it has revealed the weakness and strengths of such a system being used in healthcare institutions.[10,15,20]

The results of this study have provided some evidence concerning the weaknesses and flaws of all evaluated LIMS systems. The flaws found in the LIMSs reveal the fact that system vendors have ignored a number of information components required by standards.

Considering the standards in the design process of laboratory information systems, and evaluation of these systems based on the standards, is central to improvement of the systems’ quality. This study shows that the status of the evaluated information systems compared to the standards is unsatisfactory. Many of the information components used by the standard have been neglected in the design of these systems. It seems that lack of a national standard leads to using poor quality systems in hospitals, as also disparity among systems provided by different vendors.

Taking the difference among various software products with regard to conformity with standards and constriction of the financial resources of the healthcare institutions, it is recommended that conformity with the required standards be given a high priority when buying or developing information systems. Moreover, considering the potential disconformities with the standards, the hospitals should contemplate about a mechanism that makes the post-implementation evaluation and promotion of the systems possible. In addition, prior to designing and implementing the system, the standards proposed by the certified institutions and organizations must be taken into account. Furthermore, prior to and during the design and implementation processes of the system, requirement analyses from the stockholders and potential users of the system should be carried out. Finally, it is recommended to provide required training to all the users of the system and to conduct periodical evaluation of the system, to prevent latent problems.


Given the results of the study, it can be claimed that compared to the standards, none of the LIMSs used by the hospitals of the city of Isfahan possess a good position. Based on the results, it seems that the software provider companies did not pay due attention to the appropriate informational components suggested in the standards. Hence, it is suggested that when designing and creating the system, the standards presented by the valid organizations and institutes must come into focus, too.


The authors would like to thank the personnel of all the studied systems for their assistance


Source of Support: Deputy of research informatic of medical sciences university

Conflict of Interest: None declared


1. Tagger B. An introduction and guide to successfully implementing a LIMS (laboratory information management system) 2011. [Last accessed on 2013 Jul 1]. p. 1. Available from: .

2. McClatchy KD. 2nd ed. Philadelphia: Lippincott Williams and Wilkins; 2002. Clinical Laboratory medicine; pp. 122–137.

3. Wolper LF. 4- ed. Sudbury, MA: Jones and Berlet; 2004. Health Care Administration: Planning Implementation and Managing Organization Delivery System; p. 688.

4. Douglas S, Robert A, Nash A Validation Approach for Laboratory Information Management System. Journal of Validation Technology. 2002;9(1):6–14.

5. Martin R, Ralph PH, Krishnamurthy TR. Bellagio Italy: Georgia Institute of Technology; 2008. Laboratory information management systems in resource-Limited Environments.

6. Gimbel S, Micek M, Lambdin B, Lara J, Karagianis M, Cuembelo F, et al. An assessment of routine primary care health information system data quality in Sofala Province, Mozambique. Popul Health Metr. 2011;9:12.[PMC free article][PubMed]

7. World Health organization. Developing Health Management Information Systems A practical guide for developing countries. World health organization regional office the western pacific. 2004. [Last accessed on 2013 Jul 1]. Available from: .

8. Karimi S, Saghaeiannejad Isfahan S, Farzandipour M, Esmaeili Ghayaomabadi M. Comparative study of minimum data sets of health information management of organ transplantation in selected countries and presenting appropriate solution for Iran. Health Information Management Journal. 2011;7:497–505.

9. Hoseini A, Moghadasi H, Jahanbakhsh M. Designing minimum data sets of diabetes mellitus: Basis of effectiveness indicators of diabetes management. Health Inf Manage. 2010;7:340.

10. Ahmadi M, Rafiei F, Habibi Koulaei M, Mirkarimi A. A comparison of data elements of nursing minimum data set. Iranian Journal of Nursing Resaerch. 2012;7:45–52.

11. Ahmadian L, Cornet R, Kalkman C, Keizer NF. Development of a national core dataset for respective assessment. Methods Inf Med. 2009;48:155–61.[PubMed]

12. Ahmadian L, van Engen-Verheul M, Bakhshi-Raiez F, Peek N, Cornet R, de Keizer NF. The role of standardized data and terminological systems in computerized clinical decision support systems: Literature review and survey. Int J Med Inform. 2011;80:81–93.[PubMed]

13. Health Information Technology Organization. American National Standards Institute. [Last accessed on 2012 Jul]. Available from: .

14. Bevan N. Quality in use: Meeting user needs for quality. Journal of Systems and Software. 1999;49:89–96.

15. Farzandipour M, Sadoughi F, Meidani Z. Hospital information systems user needs analysis: A vendor surveys. Journal of Health Informaics in Developing Countries. 2011:147–54.

16. Thyvalikakath TP, Monaco V, Thambuganipalle HB, Schleyer T. A usability evaluation of Four Commercial dental computer-based patient record system. J Am Dent Assoc. 2008;139:1632–42.[PMC free article][PubMed]

17. Khajouei R, Peek N, Wierenga PC, Kersten MJ, Jaspers MW. Effect of predefined order sets and usability problems on efficiency of computerized medication ordering. Int J Med Inform. 2010;79:690–8.[PubMed]

18. Khajouei R, De Jongh D, Jaspers MW. Usability evaluation of a computerized physician order entry for medication ordering. Stud Health Techno Inform. 2009;150:532–6.[PubMed]

19. Khajouei R, Jaspers MW. The impact of CPOE medication systems’ design aspects on usability, workflow and medication orders: A systematic review. Methods Inf Med. 2010;49:3–19.[PubMed]

20. Ahmadi M, Habibi KM. Status of Nursing Information Systems in Iran. Hakim Research journal. 2010;13:185–91.

21. Azizi A, Hajavi A, Haghani H, Shojaei Baghini M. Respect rate of hospital information system criteria of american college of physicians in educational hospitals of Iran, Tehran and Shahid Beheshti medical sciences. Journal of Health Information Management. 2010;7:323–9.

Articles from Journal of Education and Health Promotion are provided here courtesy of Wolters Kluwer -- Medknow Publications

Categories: 1

0 Replies to “Lims Evaluation Essay”

Leave a comment

L'indirizzo email non verrà pubblicato. I campi obbligatori sono contrassegnati *