Perceptions 2011: An International Survey of Library Automation

by Marshall Breeding. January 28, 2012

Launch the interactive version of the survey's statistical results

Introduction

The ability for libraries to carry out their operations efficiently and effectively depends—at least to a certain extent—on having an automation system well suited to their needs. In any given year, we see a certain level of churn of libraries migrating from automation systems deemed obsolete or ill-fitting to their current expectations, moving on to new alternatives. The pool of choices for new automation products spans a set of diverse options, though within a limited pool, narrowed by the continual process of industry consolidation. Libraries in immediate need of replacing their current system, or in the process of making longer term technology strategies, need data from a variety of sources as they evaluate options. Technical documentation, marketing materials, product demonstrations, product vision statements and functionality checklists represent some sources of information to help libraries evaluate automation products.

Another important avenue of investigation involves data from libraries involved with any given product that describes their first-hand experience. This survey aims to measure the perceptions libraries have regarding their current automation products, the companies that support them and to capture their intentions about future migration options. It also explores interest in open source library automation systems, a key issue for the industry. This survey aims to gather sufficient numbers of responses, reflecting the subjective experience of any given library to create meaningful results, reasonably informative about the collective experience of libraries with this set of products and companies.

This year’s edition of the survey provides the ability to view the response data not just in a broad aggregation, but to also according to factors such as the library type (public, academic, school, etc) and by the collection size. By looking at these subsets we are able to derive more nuanced conclusions. In the broad results some of the products that serve very small libraries dominate as top performers. A different dynamic emerges when considering specific demographic categories. We provide some examples of the category combinations possible in the tables placed into this report. We have also provided the tools to apply these factors across all of the categories in the interactive version of the survey results.

Some libraries may refer to the results of this survey as they formulate technology strategies or even consider specific products. I urge libraries not to base any decision solely on this report. While it reflects the responses of a large number of libraries using these products, I hope this survey serves more as an instrument to guide what questions that a library might bring up in their considerations and not to make premature conclusions based on subjecive responses. Especially for libraries with more complex needs, it's unrealistic to expect satisfaction scores at the very top of the rankings. Large and complex libraries exercise all aspects of an automation system and at any given time may have outstanding issues that would naturally result in survey responses short of the highest marks.

The survey results also aim to provide useful information to the companies involved in the library automation industry. While each company likely performs its own measures of client satisfaction, this survey may show perceptions in different ways. I hope that the rankings in each category and the published comments provide useful information to help each of the companies hone in on problem areas and make any needed adjustments to their support procedures or product directions.

This year marks the fifth annual edition of the survey, with 2432 responses, more than any previous year (2010 = 2,173, 2009=2,099, 2008=1,453, 2007=1,779). In very broad terms, the survey results are similar this year, but with some interesting new trends. Libraries of all sizes responded, including 251 will collections over 1 million items, 1,189 with collections over 100,000, 1,083 with collections under 100,000, 750 with collections under 750, and 376 libraries with under 20,000 items. There were 131 responses with no collection size provided.

Top survey findings

  • Apollo, developed by Biblionix topped the rankings in ILS satisfaction, Company satisfaction, Functionality Completeness, Customer Support, and Customer Loyality. This product scored best last year in product and company satisfaction and topped all of the satisfaction catetories in the 2009 survey. Apollo finds use exclusively in small public libraries, reflected by its leading the competition on ILS satisfaction when considering only libraries with collections under 25,000 items. When lookinig at only public libraries, Apollo receives top ratings in ILS satisfaction. Most libraries adopting Apollo and have migrated from abandoned products such as Winnebago Spectrum and Athena or or are automating for the first time.
  • When considering large libraries with collections larger than one million items, Polaris received highest scores, followed by Millennium.
  • EOS.Web ranked second in the category of ILS support.
  • 566 libraries indicated that they are considering migrating to a new ILS. Innovative Interfaces Sierra was mentioned most frequently as a replacement candidate (88), followed by Evergreen (87).
  • Products that ranked highest in earlier years of the survey, including and Polaris from Polaris Library Systems, Library.Solution from The Library Corporation, AGent VERSO from Auto-Graphics, continue to receive satisfaction scores just as high as before, but fall below the superlative marks given by libraries involved with Apollo, OPALS, or Koha as supported independently or by ByWater Solutions.
  • Companies and products serving large and complex library organizations and diverse library types receive a broader range of responses, and fall into a middle tier of rankings. Yet where they fall within this middle ground represents important differences. Millennium from Innovative Interfaces, Library.Solution from The Library Corporation, and Evergreen from Equinox Software, and came out as very strong performers at the top of this middle tier.
  • Except for the libraries already using one, the survey reflected fairly low levels of interest in migrating to an open source ILS, even when the company rates their satisfaction with their current proprietary ILS and its company as poor. Other than libraries already running an open source ILS, and for Talis Alto (5), the mode score from libraries using proprietary ILS products was 0. The mode score in this category for all the open source products was 9. Though the open source interest scores were low, a substantial portion of libraries that registered some interest in moving to a new ILS named open source products among the replacement candidates.

General Information about the Survey

The survey has been conducted annually since 2007. Previous editions continue to be available: 2007, 2008, 2009, and 2010.

This year, the survey attracted 2432 responses from libraries in 77 different countries. The countries most strongly represented include the United States (1,756 responses), followed by Canada (160), United Kingdom (104), Australia (104) and New Zealand (30). As with the general demographics of the lib-web-cats database, the respondents of the library primarily come from libraries in English-speaking countries. Survey results were gathered between November 14, 201 and 1January 27, 2012 (Full demographic summary).

The survey attracted 50 or more responses from libraries using Millennium (458), Symphony (333), Voyager (154), Horizon (150), ALEPH 500 (140), Library.Solution(132), Polaris (106), OPALS (81), AGent VERSO (67). Koha -- ByWater Solutions (62), Apollo (54), Evergreen -- Equinox Software (52), Many other products were represented in the survey with few number of responses. Systems with less than 15 did not appear in the main statistical tables. These responses can be seen through the individual ILS Product Reports available.

This article is an original publication of Library Technology Guides and is not slated to appear in any print publication. Please direct any comments or enquiries to the author.

This survey and its analysis reflect my ongoing interest in following trends in the library automation industry. It is designed to complement, and not replace, the annual Automation Systems Marketplace feature that I have written the last seven years for Library Journal. The survey underlying the Library Journal article relies on information provided by the companies that offer library automation products and services. The survey that serves as the basis for this article collects data from the libraries themselves.

Survey Results

Statistics related to the question: How satisfied is the library with your current Integrated Library System (ILS)?

Satisfaction Score for ILS Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Apollo52 1 4 10 37 98.609 1.25
OPALS79 1 3 2 7 26 40 98.209 1.01
Koha -- Independent31 1 2 5 12 11 87.978 1.62
EOS.Web20 1 4 10 5 87.958 1.57
Polaris102 4 1 3 3 19 37 35 87.778 0.69
AGent VERSO67 1 1 2 6 12 21 24 97.728 0.86
Atriuum30 1 3 1 7 7 11 97.638 1.28
Koha -- ByWater Solutions59 1 1 3 3 13 23 15 87.618 0.26
Library.Solution129 2 3 3 5 12 33 31 40 97.458 0.70
Destiny37 1 1 2 1 5 10 8 9 76.977 1.48
Spydus21 1 6 8 6 76.907 1.31
Millennium454 7 2 4 10 13 37 57 141 128 55 76.887 0.42
Virtua35 1 1 2 3 3 8 15 2 86.807 1.18
Evergreen -- Equinox Software50 1 1 1 4 2 9 18 9 5 76.527 1.27
ALEPH 500137 1 2 3 7 11 11 22 46 27 7 76.287 0.60
Symphony (Unicorn)316 4 6 9 11 20 37 58 101 55 15 76.187 0.39
Voyager154 1 4 17 5 18 27 56 22 4 76.077 0.40
Circulation Plus16 3 2 2 2 1 4 2 86.006 0.75
Horizon149 2 4 6 5 11 28 22 44 17 10 75.916 0.49
Koha -- LibLime47 5 1 3 2 3 5 6 9 9 4 75.476 1.02
Talis Alto18 1 1 3 3 6 3 1 65.286 1.41
All Responses2378 38 24 44 85 95 204 301 646 548 393 76.747 0.18


ILS Satisfaction for Public Libraries:

Satisfaction Score for ILS Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Apollo52 1 4 10 37 98.609 1.25
Polaris87 2 1 3 3 18 30 30 87.808 0.75
Koha -- ByWater Solutions39 1 2 1 9 13 13 87.798 1.28
AGent VERSO50 1 1 1 3 9 16 19 97.768 0.99
Atriuum28 1 3 1 7 6 10 97.578 1.32
Library.Solution99 2 3 2 2 10 24 27 29 97.438 0.80
Spydus17 5 7 5 77.007 1.46
Millennium145 2 2 3 7 6 25 37 42 21 86.937 0.75
Evergreen -- Equinox Software47 1 1 1 3 2 8 18 9 4 76.537 1.17
Destiny22 1 2 1 4 7 4 3 76.507 1.92
Symphony (Unicorn)146 4 1 6 7 8 11 24 46 31 8 76.217 0.66
Horizon100 2 4 3 2 8 17 18 32 8 6 75.836 0.60
Koha -- LibLime21 2 1 2 4 3 3 4 2 55.626 1.53
All Responses1096 23 12 20 32 47 71 148 276 247 220 76.817 0.27


ILS Satisfaction for Academic Libraries:

Satisfaction Score for ILS Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Koha -- Independent16 3 5 8 98.319 2.25
Virtua23 1 1 3 5 12 1 87.048 1.46
Millennium237 4 2 1 5 6 26 24 81 61 27 76.817 0.32
ALEPH 500101 1 2 2 6 8 10 16 30 22 4 76.187 0.70
Horizon31 2 2 3 7 8 6 3 76.067 1.26
Voyager117 1 2 14 5 15 17 43 17 3 76.047 0.46
Symphony (Unicorn)110 3 3 3 9 17 20 40 10 5 76.047 0.67
All Responses777 13 10 13 42 39 93 95 239 161 72 76.397 0.25


ILS Satisfaction for Very Large Libraries (Collection over 1 million items)

Satisfaction Score for ILS Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Polaris16 1 1 7 7 88.258 1.75
Millennium83 1 1 3 3 4 9 30 26 6 76.877 0.77
ALEPH 50038 5 3 6 14 7 3 76.637 1.14
Symphony (Unicorn)40 3 1 2 7 8 13 6 75.976 0.79
Voyager29 5 1 4 7 8 4 75.836 0.56
Horizon15 1 3 2 3 4 2 64.475 1.55
All Responses251 2 2 7 11 14 24 36 76 60 19 76.477 0.38


ILS Satisfaction for Medium-sized Libraries (Collection over 200,000)

Satisfaction Score for ILS Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Polaris38 1 1 6 16 14 88.038 1.14
Library.Solution20 1 1 1 2 5 10 97.959 1.79
Millennium264 3 2 2 7 5 22 31 88 79 25 76.887 0.49
Virtua20 1 3 3 5 8 86.807 0.89
ALEPH 50086 2 2 5 6 6 15 29 16 5 76.297 0.75
Symphony (Unicorn)143 2 3 6 2 8 27 31 43 18 3 75.926 0.33
Voyager87 1 14 4 11 14 31 9 3 75.906 0.54
Horizon72 1 3 3 4 7 10 13 21 7 3 75.656 0.71
All Responses859 11 11 16 38 41 89 129 258 185 81 76.477 0.20


ILS Satisfaction for Small Libraries(Less than 25,000 items)

Satisfaction Score for ILS Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Apollo28 1 2 5 20 98.579 1.70
OPALS61 2 1 6 19 33 98.319 1.15
Koha -- Independent15 1 2 6 6 87.938 2.07
AGent VERSO31 2 3 4 11 11 87.848 1.44
Koha -- ByWater Solutions19 1 1 4 8 5 87.688 1.61
Destiny21 1 1 1 1 4 6 7 97.298 1.96
Millennium29 2 1 2 5 6 7 6 86.767 1.49
Symphony (Unicorn)28 1 1 3 7 7 7 2 66.507 1.51
All Responses463 12 3 6 12 12 29 49 92 112 136 97.138 0.14



Statistics related to the question: How satisfied is the library overall with the company from which you purchased your current ILS?

Satisfaction Score for Company Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Apollo52 1 2 11 38 98.659 1.25
OPALS80 1 1 7 23 48 98.459 1.01
EOS.Web20 2 13 5 88.158 1.79
Koha -- ByWater Solutions59 2 1 1 8 18 29 98.078 0.26
AGent VERSO67 1 1 7 5 23 30 98.048 1.10
Polaris100 1 2 3 2 4 15 36 37 97.808 0.80
Atriuum30 2 1 3 3 10 11 97.708 1.46
Library.Solution129 1 1 1 2 10 11 24 34 45 97.568 0.70
Koha -- Independent26 1 1 3 4 8 9 97.358 1.57
Destiny37 1 3 6 10 8 9 77.247 1.48
Circulation Plus16 1 4 1 2 3 5 97.008 2.25
Evergreen -- Equinox Software50 2 3 2 4 4 5 21 9 86.908 1.27
Virtua35 1 6 5 9 11 3 86.807 1.18
Spydus21 1 2 7 7 4 66.527 1.31
Millennium453 6 6 16 16 22 49 58 111 122 47 86.507 0.42
Voyager154 1 1 4 10 12 16 22 54 27 7 76.217 0.24
ALEPH 500137 2 2 7 8 12 12 23 38 21 12 76.047 0.43
Horizon149 2 2 9 9 15 12 27 36 29 8 75.946 0.49
Symphony (Unicorn)316 5 7 18 20 24 34 57 80 51 20 75.886 0.39
Talis Alto18 1 3 7 4 3 55.175 1.41
Koha -- LibLime46 6 3 6 3 3 4 5 4 9 3 84.635 1.18
All Responses2364 40 26 78 93 121 207 297 486 563 453 86.667 0.19



Statistics related to the question: How complete is the functionality of this ILS relative to the needs of this library?

ILS Functionality Score Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Apollo51 1 3 1 17 29 98.379 1.26
OPALS79 1 2 5 19 17 35 97.958 1.01
Polaris102 1 4 8 20 46 23 87.718 0.59
Koha -- Independent31 1 1 1 9 9 10 97.618 1.62
EOS.Web20 1 2 7 6 4 77.508 1.34
AGent VERSO67 2 3 7 16 22 17 87.498 0.86
Atriuum30 2 2 2 7 11 6 87.378 1.46
Library.Solution127 3 4 9 11 31 46 23 87.318 0.71
Koha -- ByWater Solutions59 1 1 3 9 17 17 11 77.257 0.13
Millennium451 3 3 5 7 18 29 67 144 116 59 76.927 0.42
Virtua35 1 1 2 1 5 10 14 1 86.777 1.18
Spydus21 1 3 4 7 5 1 76.717 1.31
Destiny36 1 1 1 1 3 6 12 5 6 76.647 1.50
Symphony (Unicorn)310 1 2 7 17 17 30 60 91 67 18 76.397 0.40
Evergreen -- Equinox Software49 1 1 2 3 3 8 20 8 3 76.297 1.29
ALEPH 500133 3 8 8 10 8 15 41 31 9 76.237 0.61
Horizon141 1 2 6 11 8 21 24 36 20 12 76.016 0.59
Voyager152 1 8 16 7 16 26 44 31 3 76.007 0.32
Koha -- LibLime45 2 1 4 3 3 2 6 16 6 2 75.647 1.04
Talis Alto16 1 2 3 3 6 1 75.636 1.50
Circulation Plus15 2 1 2 4 1 4 1 55.535 0.52
All Responses2334 17 24 57 98 98 174 335 632 578 321 76.707 0.19



Statistics related to the question: How satisfied is this library with this company's customer support services?

Satisfaction Score for ILS Support Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Apollo52 4 9 39 98.679 1.25
EOS.Web20 3 7 10 98.359 2.01
OPALS80 1 1 3 6 19 50 98.359 1.01
AGent VERSO67 1 1 1 2 1 19 42 98.349 1.10
Koha -- ByWater Solutions59 1 2 3 7 11 35 98.179 0.65
Atriuum28 2 3 1 2 9 11 97.648 1.51
Polaris100 2 2 4 2 6 22 29 33 97.558 0.70
Library.Solution129 2 4 3 8 11 20 33 48 97.528 0.70
Circulation Plus16 5 2 3 6 97.318 2.25
Destiny37 1 1 1 2 5 5 12 10 87.248 1.48
Evergreen -- Equinox Software50 2 1 3 5 7 5 17 10 86.868 1.27
Virtua35 1 1 2 4 3 11 9 4 76.697 1.18
Spydus21 1 1 7 7 5 66.677 1.31
Koha -- Independent26 3 1 1 1 1 4 9 6 86.628 1.57
Millennium451 5 8 5 9 30 52 68 114 115 45 86.557 0.42
Horizon147 3 3 6 12 13 23 29 40 18 86.557 0.41
Symphony (Unicorn)314 4 10 10 14 18 42 53 70 65 28 76.167 0.45
Voyager152 3 7 11 5 18 23 47 30 8 76.167 0.16
ALEPH 500137 3 1 7 7 10 16 20 40 21 12 76.077 0.34
Talis Alto18 2 1 7 4 4 55.395 1.18
Koha -- LibLime47 3 6 7 4 4 7 5 9 2 84.665 1.17
All Responses2350 39 38 58 78 114 211 290 466 557 499 86.747 0.19



Where does this library direct most of its ILS support issues?

ProductILS VendorThrough another library or consortiumTotal Responses
Evergreen -- Equinox Software123552
Koha -- ByWater Solutions381862
Koha -- LibLime321348
Koha -- Independent51731
Millennium34794458
Symphony21794333
Apollo50054
OPALS59681
Polaris7722106
EOS.Web20020
AGent VERSO501267
Atriuum28132
ALEPH 5009045140
Voyager10445154



Statistics related to the question: How likely is it that this library will purchase its next ILS from this company?

Loyalty to Company Score Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Apollo52 1 1 1 5 44 98.699 1.25
OPALS78 2 1 4 2 12 57 98.469 1.02
EOS.Web20 1 1 6 12 98.359 1.12
Koha -- ByWater Solutions59 2 2 2 3 3 9 38 97.989 0.00
Polaris100 1 2 2 2 2 2 9 28 52 97.959 0.80
AGent VERSO66 2 1 1 3 3 8 18 30 97.708 0.86
Atriuum29 1 5 2 5 7 9 97.348 1.30
Library.Solution128 5 4 4 4 3 9 7 12 25 55 97.058 0.71
Spydus21 1 1 4 8 5 2 77.007 1.31
Koha -- Independent24 3 4 1 6 10 96.888 1.63
Evergreen -- Equinox Software48 5 2 1 2 2 3 5 9 19 96.698 1.30
Millennium452 13 5 16 18 24 52 50 79 93 102 96.557 0.42
Destiny37 3 2 2 3 3 9 4 11 96.467 1.48
Virtua35 2 1 3 1 4 3 8 8 5 76.237 0.51
Voyager154 5 2 12 10 30 17 34 24 20 76.107 0.40
Circulation Plus16 1 1 2 4 2 2 4 55.947 1.25
ALEPH 500135 9 10 3 5 6 24 10 20 29 19 85.767 0.43
Horizon149 10 5 8 5 12 17 21 29 23 19 75.726 0.57
Symphony (Unicorn)313 20 16 16 12 31 44 40 52 54 28 85.486 0.45
Talis Alto18 1 1 2 1 4 3 4 2 55.226 1.18
Koha -- LibLime47 11 4 4 2 1 3 5 5 7 5 04.325 1.17
All Responses2344 132 66 68 75 125 263 204 340 427 644 96.427 0.19



Statistics related to the question: Has the customer support for your ILS gotten better or gotten worse in the last year?

Change in customer support quality Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Apollo49 1 1 5 3 3 9 27 97.889 1.29
OPALS75 1 9 4 5 28 28 87.798 0.92
AGent VERSO66 1 1 9 8 5 13 29 97.538 0.98
Koha -- ByWater Solutions56 1 11 6 4 9 25 97.438 0.67
Library.Solution128 3 2 3 8 22 9 21 23 37 96.827 0.71
Atriuum29 2 8 2 5 6 6 56.797 0.93
Virtua34 1 10 3 8 6 6 56.767 1.20
EOS.Web20 6 3 5 2 4 56.757 1.34
Koha -- Independent24 1 2 1 5 4 6 5 86.427 1.63
Polaris97 3 1 2 2 6 23 10 12 18 20 56.377 0.51
Evergreen -- Equinox Software48 2 1 1 16 6 4 12 6 56.276 1.30
Destiny35 1 1 2 14 2 4 4 7 56.175 1.52
Circulation Plus15 4 4 1 1 2 3 46.135 1.29
Horizon147 1 1 6 3 12 49 17 23 15 20 55.976 0.41
Symphony (Unicorn)313 3 10 10 12 27 80 36 58 46 31 55.896 0.45
Millennium437 4 5 8 14 40 178 61 56 50 21 55.635 0.43
ALEPH 500137 2 2 4 7 16 50 14 12 15 15 55.615 0.34
Voyager149 2 2 4 7 13 62 16 19 15 9 55.525 0.41
Spydus20 3 10 2 4 1 55.505 1.12
Koha -- LibLime46 2 1 3 4 6 13 4 4 6 3 55.115 0.74
Talis Alto18 1 4 3 8 2 54.445 0.71
All Responses2301 48 29 50 73 170 691 252 298 331 359 56.076 0.19



Statistics related to the question: How likely is it that this library would consider implementing an open source ILS?

Interest Level in Open Source Response Distribution Statistics
CompanyResponses 0 1 2 3 4 5 6 7 8 9 ModeMeanMedianStd Dev
Koha -- Independent22 1 1 1 2 17 98.419 1.92
Koha -- ByWater Solutions49 2 1 1 2 2 3 38 98.069 0.86
Evergreen -- Equinox Software38 3 1 1 2 1 2 1 27 97.539 1.46
OPALS52 6 1 1 2 2 1 1 38 97.319 1.25
Koha -- LibLime41 7 1 1 2 1 1 4 2 22 96.349 1.25
Talis Alto18 1 1 2 3 4 4 1 2 54.505 1.18
Voyager152 25 16 16 16 12 21 17 13 10 6 03.794 0.41
Horizon146 31 15 16 11 10 16 14 11 12 10 03.754 0.25
Symphony (Unicorn)310 62 29 45 21 34 45 18 27 12 17 03.513 0.11
Millennium447 100 48 64 32 35 52 37 30 21 28 03.413 0.19
ALEPH 500137 31 14 20 7 13 20 15 6 3 8 03.323 0.51
Circulation Plus16 4 2 2 1 3 2 2 03.313 1.75
Destiny37 7 6 7 5 1 3 1 3 1 3 03.142 0.82
Library.Solution127 36 11 20 12 10 11 10 8 2 7 02.972 0.18
Virtua35 10 4 8 2 3 2 3 1 2 02.712 0.85
AGent VERSO66 18 12 7 4 9 8 4 3 1 02.592 0.62
Atriuum30 9 6 2 2 5 3 1 1 1 02.502 0.00
Apollo50 21 7 4 3 2 6 2 2 1 2 02.281 0.00
Spydus21 7 4 2 2 1 3 1 1 02.241 1.31
EOS.Web20 11 2 1 1 2 1 1 1 01.650 1.57
Polaris99 46 12 20 8 2 6 3 1 1 01.481 0.10
All Responses2273 524 227 275 148 165 245 149 138 104 298 03.733 0.08


Open Source interest scores of libraries currently using an open source ILS compared to 2010 results:

Product20112010
Koha – Independent8.688.41
Koha – ByWater Solutions9.008.06
Koha – LibLime8.506.34
OPALS 8.327.31
Evergreen8.317.53

Percentages of libraries indicating that they are considering migrating to a new ILS

Product20112010200920082007Comments
Millennium31.2218.7311.718.286.6965 Libraries using Millennium responded that they are considering Sierra; Evergreen = 25; Koha = 13
Symphony22.3920.2115.8123.0814.58
Horizon54.0057.3045.6961.0649.4539 out of 90 libraries indicated that SirsiDynix Symphony was one of the candidates for migration
Aleph25.7118.8711.8512.129.0914 libraries included Alma as a migration candidate.
Voyager38.3132.2618.9021.8421.6221 libraries included Alma as a migration candidate.
Polaris1.895.776.529.431.56
Apollo0.000.000.000.000.00
Evergreen1.920.000.000.000.00


Summaries of replacement candidates for libraries indicating they are considering migrating to a new ILS

Migration Trends
Innovative Interfaces Sierra 88
Evergreen 87
Koha74
Polaris 69
SirsiDynix Symphony67
Ex Libris Alma51
OCLC WorldShare53
Kuali OLE16
Follett Destiny7
OPALS4
Total considering migration566


An interactive version of the statistical reports, is available which includes the ability to view the responses for each of the ILS products, along with the redacted comments.


ILS Turnover Reports

Another set of reports provide information on the ILS products that were selected during 2011 by libraries registered in lib-web-cats. [Note: these numbers are not comprehensive.]

The ILS Turn-over report counts and lists the automation systems recorded as selected or installed in 2010 with a breakdown of the previous systems displaced.

The Reverse ILS Turn-over report. counts and lists the automation systems recorded as replaced in 2011 with a breakdown of the new systems that were selected

General Observations

In one of the more interesting observations of the survey, respondents tend to blur their perceptions across many of the questions. Those with generally positive satisfaction, tend to answer the questions about ILS satisfaction, company satisfaction, support satisfaction, and ILS functionality without major distinction. Few responses will indicate high satisfaction in one question and moderate or low satisfaction in others. Despite this tendency, the variation between questions does result in some differences in rankings in the companies across categories. The blurring across categories is especially conspicuous in the question asking whether service has improved in the last year. Aggregate responses in this category tend to be quite high for products with strong satisfaction. Taken literally, high numbers in this category imply that great improvement had taken place since the previous year. Numbers in the middle range would indicate that support had maintained an existing level of high performance. Since this question was introduced last year, it does not seem that it has been especially successful in measuring changes in support performance.

Small libraries give measurably more positive responses on the survey than larger libraries, bearing out a trend that the size and complexity have a downward impact on overall satisfaction with automation scenarios. Larger libraries press the limits of functionality and encounter more support issues for which there may not be an easy answer or immediate response possible. Across all libraries with collections less than 25,000 items, the average satisfaction score for their ILS was 7.13, compared to 6.47 for both the groups of medium-sized and very large libraries. The top performers in the small library group received superlative ratings: Apollo with 8.57 and OPALS at 8.31; in the large library arena top scoring Polaris averaged ILS satisfaction of 8.25 and second-place Millennium at 6.87. In the middle tier of libraries Polaris received 8.03 followed by Library.Solution at 7.95.

This survey has not attracted a great deal of interest from the K-12 school library arena, with only 126 out of the 2,432 total responses. Of these, 57 were from libraries using OPALS, 14 from Destiny, 11 using some version of Koha, 8 from Library.Solution, 5 using Millennium, 4 using SirsiDynix Symphony, with a few other ILS products represented in smaller numbers. Destiny holds dominant market share among school libraries. OPALS represents a smaller portion of the K-12 school market, though is much more strongly represented in this survey. Libraries using OPALS express superlative perceptions of the software and for support arrangements.

Perceptions Open Source ILS Products and Support Companies

Consistent with results from previous years, the scores in the "Interest Level in Open Source" naturally run high for those libraries already involved with an open source ILS, ranging from a 8.41 given by those running Koha implemented independently, 8.06 when supported by ByWater Solutions, through 7.53 for Evergreen. Libraries using LibLime’s version of Koha reported a bit less enthusiasm for open source ILS with a mean response score of 6.34. For those libraries running a proprietary ILS, the interest in open source ILS seems mostly indirectly proportional to satisfaction with the ILS, company, and support. Libraries running proprietary products that rate high satisfaction with the ILS, company, and support categories selected lower levels of interest in open source alternatives, while those more dissatisfied show at least some higher interest. The scale of interest in open source from those running proprietary systems tops out at 4.50 (Talis Alto) compared to scores greater than 6.34 from existing open source practitioners.

Overall there was a slight softening in the enthusiasm for open source relative to last year, with mean scores of the question probing interest in open source ILS dropping since 2010. In the case of LibLime, the drop was significant, down to 6.34 in 2011 from 8.50 in 2010.

The survey does not support the notion that libraries running open source are generally more satisfied than those using proprietary systems. Among small libraries, Apollo, a proprietary product offered through software as a service beats any of the open source products, though the open source OPALs ranks only a tiny notch below. Open source and proprietary products are both dispersed throughout the result tables. Neither proprietary nor open source products dominate the top rankings. In the new question category added this year asking about the completeness of functionality, we saw some trends rating the open source products slightly downward and proprietary ones slightly upward. For example, Evergreen ranked fourteenth in overall ILS satisfaction and fifteenth in functionality, SirsiDynix Symphony ranked sixteenth in ILS satisfaction and fourteenth in functionality; Koha supported by ByWater Solutions was perceived as ninth in functionality though eighth in overall satisfaction; Millennium placed twelfth in overall satisfaction and tenth in perceived functionality; Polaris ranked fifth in overall ILS satisfaction and third in functionality.

Libraries that have implemented Koha independently reflect higher satisfaction than those that rely on commercial support arrangements. One might suppose that this reflects their enthusiasm toward open source and that they are essentially evaluating themselves rather than an external organization. ByWater Solutions continues to reap high praise from their support customers, though their scores dropped a fraction since last year. Libraries using the versions of Koha supplied and supported by LibLime reflected some of the lowest satisfaction scores on this year's survey across all categories. Satisfaction with Evergreen as supported by Equinox Software fell into the middle tier of results in most categories.

It’s important to note that despite any softening in the scores rating interest in open source ILS products, many libraries indicating that they are interested in a migration mention open source products as a replacement candidate: Evergreen (87) Koha (74)and Kuali OLE (16).

Comments on the Products and Companies specializing in the Automation of Smaller Libraries

We noted above that small libraries tend to give higher satisfaction ratings than do larger libraries. 463 of the total of responses came from librareis with 25,000 or fewer items in their collections. Products that appeared only in the small library category included: Apollo, OPALS, AGent VERSO, Koha implemented independently or with support from ByWater Solutions, and Destiny. Some small libraries use products generally oriented toward larger libraries such as Millennium or SirsiDynix Symphony, but as members of a consortium.

Apollo, a hosted ILS provided by Biblionix used by small public libraries, received top rankings for ILS satisfaction (8.60), company satisfaction (8.65); ILS functionality (8.37), customer support (8.67); and company loyalty (8.69). The number of responses for Apollo (53) was down this year relative to previous years. No libraries using Apollo indicated consideration of moving to a new ILS, none plan for a discovery interface, and 51 out of 54 report on-schedule installation. Comments offered from libraries using Apollo were overwhelmingly positive.

AGent VERSO from Auto-Graphics received positive ratings in each of the categories. The average ILS satisfaction ratings for this product improved over last year, moving up to 7.72 from 7.40 last year. Perceptions for ILS satisfaction ranked sixth, support ranked Fourth (8.34), sixth in ILS functionality (7.49) sixth for loyalty to company (7.70). Libraries using AGent VERSO ranked very low in interest to an open source ILS (2.59). Overall survey results confirm very high perceptions for AGent VERSO as an ILS and Auto-Graphics as a company and comments offered mostly reflect high praise, though some reflect some transition issues related to the company's new Iluminar interface.

Comments on the Products and Companies specializing in the Automation of Larger Libraries

Polaris tops the rankings within its sector of larger public libraries. This product has attracted very positive rankings from its users across all five years of this survey. In the first years of the survey, Polaris earned top placement. A set of products has emerged in the survey in recent years that serve smaller libraries with even higher ratings. But the performance of Polaris has remained remarkably constant (ILS Satisfaction: 2011=7.77, 2011=7.77, 2009=7.79, 2008=7.73, 2007=7.78). While Polaris’ ranking in the general pool of responses for ILS satisfaction has slipped to fifth place this year, its performance has in no way diminished. When limiting the results to public libraries, Polaris ranks second. In the category of libraries with collections over one million items, Polaris ranks first, as it does among libraries with collections over 200,000 items. Only 1.89 percent of libraries using Polaris indicate interest in migrating to a new ILS.

Millennium from Innovative Interfaces, Inc. received quite respectable rankings, generally placing near the top of the middle tier of most categories. Libraries running Millennium responded to the survey in higher numbers than any other ILS (454). ILS satisfaction for Millennium slipped a small fraction since last year (ILS satisfaction: 2011=6.88, 2010=7.11, 2009=7.13, 2008=7.08, 2007=7.17). The number of libraries running Millennium indicating that they are considering migrating to a new ILS is up sharply this year (31.22 percent) compared to last year (18.73 percent), but sixty-five of these libraries indicate consideration of Innovative's own Sierra platform. Twenty-five show interest in Evergreen, with another thirteen including Koha as a migration candidate. The comments offered were diverse: common themes included complaints about system costs, others spoke highly the new Sierra platform.

Libraries using Library.Solution from The Library Corporation responded with rankings that generally fall around the top of the middle tier. In the ILS Satisfaction category, the 7.45 mean rating was ninth best; company satisfaction (7.56) and support satisfaction (7.52) scores earned similar placement. Support satisfaction for Library.Solution has improved each of the five years of the survey. Libraries gave average company loyalty ratings of 7.06 and 14.39 percent indicate interest in migrating to a new ILS.

SirsiDynix offers two ILS products represented in the survey, Symphony, (316 responses) and Horizon (149 responses). The two ILS products received remarkably similar scores. Though SirsiDynix promotes Symphony as its flagship ILS, it offers continued support and development for Horizon. Satisfaction with Symphony has remained relatively constant across all five years of the survey (ILS satisfaction: 2011=6.18, 2010=6.15, 2009=6.06, 2008=5.68, 2007=6.41). Horizon has seen a slight diminishment in satisfaction. (ILS Satisfaction: 2011=5.91, 2010=5.99, 2009=6.07, 2008=5.68, 2007=6.13). Libraries running Horizon showed an improvement in company loyalty since last year, up to 5.72 from 4.94. Interestingly, this year Symphony libraries reflect a slightly lower company loyalty score of 5.47. 54 percent of Horizon libraries and 22.52 percent of Symphony libraries indicate that they are considering migrating to a new ILS. 39 out of 90 libraries running Horizon that were considering migrating mentioned SirsiDynix Symphony a replacement option.

Two products from Ex Libris were represented in the survey results, Aleph, (139 responses) and Voyager, (154 responses); the company continues support, marketing, and developmlent for both of these ILS products. Ex Libris has announced Alma as its next generation system. Aleph and Voyager serve large and diverse organizations and offer very complex functionality, placing them into a tier of products that tend not receive superlative marks. Aleph received a mean score of 6.28 on ILS satisfaction, just a bit higher than that of Voyager (6.07). Perceptions of the company between these two products show slightly higher ratings from Voyager libraries (6.21) than those running Aleph (6.04); A simlar pattern emerged for customer support with libraries running Voyager slightly happier (6.16) than Aleph sites (6.07). 25.71 percent of Aleph libraries indicated interest in migrating to a new ILS with 14 mentioning Alma as a replacement candidate. 38.31 percent of Voyager libraries indicated interest in migrating to a new ILS with 21 mentioning Alma as a replacemetn candidate.


Details about The Survey

The survey instrument included five numerical ratings, three yes/no responses, and two short response fields, and a text field for general comments. The numeric rating fields allow responses from 0 through 9. Each scale was labeled to indicate the meaning of the numeric selection.

Four of the numeric questions probe at the level of satisfaction with and loyalty to the company or organization that provides its current automation system:

A yes/no question asks whether the library is considering migrating to a new ILS and a fill-in text field provides the opportunity to provide specific systems under consideration. Another yes/no question asks whether the automation system currently in use was installed on schedule.

view automation survey

Given the recent interest in new search interfaces, a yes/no question asks “Is the library currently considering a search interface for its collection that is separate from the ILS?” and a fill-in form to indicate products under consideration.

The survey includes two questions that aim to gauge interest in open source ILS, a numerical rating that asks “How likely is it that this library would consider implementing and open source ILS?” and a fill-in text field for indicating products under consideration.

The survey concludes with a text box inviting comments.

View the survey. (This version of the survey does not accept or record response data.)

In order to correlate the responses with particular automation systems and companies, the survey links to entries in the lib-web-cats directory of libraries. Each entry in lib-web-cats indicates the automation system currently in use as well as data on the type of library, location, collection size, and other factors that might be of potential interest. In order to fill out the survey, the responder had first to find their library in lib-web-cats and then press a button that launched the response form. Some potential respondents indicated that found this process complex.

The link between the lib-web-cats entry and the survey automatically populated fields for the library name and current automation system and provided access to other data elements about the library as needed. The report on survey response demographics, for example, relies on data from lib-web-cats.

A number of methods were used to solicit responses to the survey. E-mail messages were sent to library-oriented mailing lists such as WEB4LIB, PUBLIB, and NGC4LIB. Invitational messages were also sent to many lists for specific automation systems and companies. Where contact information was available in lib-web-cats, and automated script produced e-mail messages with a direct link to the survey response form for that library.

The survey attempted to limit responses to one per library. This restriction was imposed to attempt to sway the respondents to reflect the broad perceptions of their institution rather than their personal opinions.

The survey instrument was created using the same infrastructure as the Library Technology Guides web site—a custom interface written in perl using MySQL to store the data, with ODBC as the connection layer. Access to the raw responses is controlled through a user name and password available only to the author. Scripts were written to provide public access to the survey in a way that does not expose individual responses.

In order to provide access to the comments without violating the stated agreement not to attribute individual responses to any given institution or individual, an addition field was created for “edited comments.” This field was manually populated with text selected from the “comments” text provided by the respondent. Any information that might identify the individual or library was edited out, with an ellipse indicating the removed text. Comments that only explained a response or described the circumstances of the library were not transferred to the Edited Comments field.

Statistics

To analyze the results, a few scripts were written to summarize, analyze, and present the responses.

In order to avoid making generalizations based on inadequate sample sizes, the processing scripts included a threshold variable that would only present results when the number of responses exceeded the specified value. The threshold was set to a value of 20.

For each of the survey questions that involve a numeric rating, a set of subroutines was created to calculate and display simple statistics.

The "survey-report-by-category.pl" script processes each of the numerical ratings, displaying each of the statistical components listed above for each product that received responses above the threshold value. This report provides a convenient way to compare the performance of each ILS product for the selected question. The report sorts the statistics for each product in descending order of the mean. The report categories available correspond to the survey questions with numerical scale responses.

The “survey-product-report.pl” script provides the results for each of the ILS products mentioned in the responses. This report also provides the statistical components for each of the numeric question. It also provides the percentage of yes responses to the two yes/no questions:

[The text of this section mostly replicates what appeared in the 2007 version of this article. For for both editions of the survey I followed the same methodology for collection and and statistical analysis.]


Caveat

As I noted with previous editions of the survey, one should not read too much into the survey results. Responders to the survey provide their subjective impressions to fairly general questions. Although the survey instructions encourage responders to consider the broader institutional perceptions, it’s usually the case that multiple opinions prevail within any given library. While I believe that this survey does provide useful information about the experiences of libraries with their current integrated library systems and the companies that provide support, it should not be used as a definitive assessment tool.