Abstract
This report is a usability test and PACT analysis report of the institution library automation system. The usability test is conducted to evaluate the ease of use of the user interface of this automated interactive software system. The objective of the usability test is to learn the formal methods of a usability test of an existing system, research the features of the system, and present solutions to the problems discovered. Two usability methods: questionnaire survey and heuristic evaluation are used for the usability test. Three usability principles have been chosen to perform the library automation system usability test. It is not possible to test the system usability for all the ten Jakob Nielsen usability principles in the given time frame.
The PACT analysis is performed to determine the usefulness of the system and suggestions are made to enhance the usefulness. The PACT analysis methods used are brainstorming, research of library system documentation, user feedback, and the author’s own experience.
Scenario
An institution library has an automated management system to track the movement of the library information assets. The automated library management system provides the following facilities (Horizon Integrated Library System):
- Online Catalogue – Bibliographic, Holdings, Item, and Authority records are created by the automated system. The automated system can create bibliographic records based on Catalogue-In-Publication (CIP) information received from the publishers before the library asset is made available to the library. The library is connected to the Online Computer Library Center (OCLC) server and if the information is not available in the local database a query is submitted to the OCLC server to get information from the WorldCat database.
- Acquisition system – Library asset requests are saved and processed online, the requests can be received based on circulation information for the assets already present in the library or CIP information for the new arrivals.
- Serials System – Creation of bibliographic records for new arrivals based on CIP information received from publishers.
- User Interface – The automated software system provides a graphical user interface for the library staff and users. Different access permissions can be set for staff and users allowing access to different modules of the automated library management system. The staff can access acquisition and serials systems, whereas users can access check-in, check-out, and search interfaces.
- Inter Library Communication – Z39.50 based client-server communication interface is available for inter-library communication. The Inter-Library Loaning (ILL) facility is present.
The usefulness of the library system is measured by testing its usability and analyzing its utility. The library management system is to be evaluated for its usability, i.e. the ease of use of this system for the institution students. The utility is to be tested to measure the effectiveness of this system in enhancing student productivity. The PACT analysis is to be performed to measure the usefulness of the system. The usability testing and PACT analysis report will inform if the interactive library automated system reduces library maintenance operational cost and student training (Business benefits of Usability).
Methodology
Usability
The usability test was conducted to evaluate the following usability principles (Nielsen, 2005):
- User control and freedom – Can a student operate the library system without assistance from the library staff, the clarity of instructions in the help menu, the robustness of the system in case the student provides wrong data?
- Recognition rather than recall – when surfing through the library information how much information must be remembered by the user, does the library system support a pop-up/drop-down menu for multiple options. For example, when searching for a subject topic does the library system show other related topics.
- Help and Documentation – does the library system include a tutorial for novice users and advanced users.
- The methods used for usability testing are:
- Questionnaire survey – A questionnaire was prepared to get feedback on the library system features from fellow students.
- Heuristic evaluation – The user interface was tested to evaluate the usability principles mentioned above. Test cases were prepared and library operations: find, check-out, and check-in interfaces were tested.
Questionnaire Survey
Following questions were prepared for the survey:
After permission from the department did you have any problem in log in? This question was asked to know if the access list was configured according to the departments’ directive and if the user had to contact the library staff to get login assistance.
Were you able to issue other department information assets? This question was asked to know if access to other department assets was barred without permission.
Were you able to use the ILL service without library staff assistance? This question was asked to ascertain if the ‘help’ provided necessary instructions to ‘find’ assets on WorldCat and fill the request form for ILL.
Is the information in the bibliographic records sufficient to inform about the library asset? This question was asked to know if the CIP information appropriately described the asset. Bibliographic records are created from CIP information and this is the description of the asset a user can read online.
How often do you use a library automation system? This question was asked to ascertain the popularity of the library automation system.
Heuristic Evaluation
The author of this report conducted a first-person test of the library automation system in the following scenarios:
- Find a book on the subject of Usability. Use remote-login through the institution’s site.
- Reserve the resource online and then go to the library to perform a check-out. The asset (book) record must show in the author’s user profile.
- Return the book to the library, check-in interface is tested.
PACT Analysis
The PACT analysis of the library automation system is performed to determine the utility of this automated software interface for the users (PACT Analysis).
- People – Who are the people who can use this automated library system and what is the effect of this system on their academic life.
- Activity – What activities can a student perform on this system?
- Context – In what context is this system usable?
- Technology – What are the technologies used by the library automation system?
PACT analysis of the library automation system
People – The library automation system is to be used by the staff and students of the institution. The library automation system is in the English language; therefore the users of this system are expected to be English literate with computer knowledge. A nonuser can use the library services but will not be able to use the automated library system; he/she will have to take assistance from the library staff or fellow IT students. IT users are those who are computer literates and not necessarily students of IT-related subjects. The library has an orientation program for non-users to make them familiar with the basic library automation system features. It is an incentive for nonIT users to become IT literates. English is the teaching medium in the institution therefore it is expected that all students and staff are proficient in the English language irrespective of their nationality. The college has a small percentage of non-UK students who are from the non-English speaking country.
Activity – The objective of a library automation system is that students can easily locate the study material and issue the library assets 24hrs a day without any assistance from the library staff. The connectivity to the WorldCat database enables students to find new information resources online and take advantage of extra assets in other institutional libraries. The online library system saves student time and makes institutional libraries competitive by learning about new publications through CIP. The important activities that are performed by the library automation system are:
Allow 24hr issue and return of library assets
Allow reservation of library assets before they are physically present in the library, bibliographic records created based on CIP information informs users about forthcoming arrivals.
Allow searching in the WorldCat database, WorldCat is a library catalog of the most online libraries around the globe.
Provision for Inter-Library Loaning, the ILL system has provision to find, get permission, and reserve other online library assets. The assistance of library staff is then required to physically transfer the asset.
Provides an interface for the student to find the most appropriate information asset for his/her requirements, based on subject topic, research title, author, etc.
Context – The library automation system provides an online interface therefore students with access to the internet and a user account in the library automation system can access it remotely.
Social Context – The library automation system can be used by nonIT users to become IT aware. This system can be used by all students to relate with students in other institutions. The online library provides a chance to know which authors and publishers are referred for the subject in other institutions. In this social context, the following features of the library automation system are considered important:
- Control and freedom are provided by the user interface with online help and good documentation.
- Connectivity with other online libraries.
Physical Context – The remote accessibility to the library automation system from the internet connections provided in the student hostels saves time. A student does not have to rush to the library to submit a request for ILL; the request can be placed online. In the physical context, the following features of the library automation system are considered important:
Remote user interface.
Ability to perform remote ‘find’ in local or another online library.
Ability to submit ILL.
Organizational Context – The library automation system makes the institution staff & students IT aware and provides them an opportunity to compare with and meet international standards in all areas of education and research. This includes syllabus updates based on CIP information and other research material available online. In the organizational context the following features of the library automation system are considered important:
- IT awareness for nonIT students and staff
- OCLC interface that allows ‘find’ operation in other online libraries
- Reachability to online publishers for CIP information on latest course material publications.
Technology – The library is equipped with a sufficient number of computer systems to allow students to perform online searches of information assets. Students do not have to wait for turns on computer systems, since the library is connected to the internet, students can remote login. Backup servers and power generators keep the library system secure, up and running at all times.
Software – Library automation system supports secure connections (SSL) for remote login, username-password authentication system protects fraudulent use. An access list is configured on the firewall router that acts as a gateway to the library. Library automation system has MySQL database that provides easy to learn and use interface. The institution’s library automation system can be accessed online from the institution’s site. A z39.50 link client runs on the library server to provide connectivity with the OCLC server.
Hardware – Library automation system runs on a high-speed multi-core Intel processor server that can support multiple sessions simultaneously. The high-speed database servers both primary and backup enable fast search in a huge online database. Servers are connected through a firewall router to the internet with a high-speed always ON DSL broadband connection.
Observations
Survey
Usability Survey
The survey was conducted for 100 non-computer science students. The answers to the survey questions are:
10 out of 100 students failed to log in after the department created their user account in the library automation system. The problem was investigated and it was found that they either mistyped or were not aware that the system was case sensitive. The ‘help’ on the login page informed that username and password must be entered as given by the department, upper/lower case.
NO, all 100 students could not access other department assets. Some students suggested that this restriction should be removed.
Only 5 out of 100 students had used the ILL service and were able to ‘find’ or ‘request’ the necessary information asset.
Most students were searching for the syllabus book and were happy to find the subject, title, and author. 20 out of 100 students searched for new material that was not mentioned in the syllabus. These were either the reference books suggested by the teacher or found on the internet. They commented that bibliographic record was authorized publisher information, but different and more information was available online. The desire was to find authenticated information sources and for this, they had to rely upon physical access to the information asset.
The Library automation system was found to be very popular with all students. The noncomputer science students had great ideas about the user interface and were familiar with most features of the library automation system.
Heuristic Evaluation
Following are the observations of the heuristic evaluation of the library automation system:
User control and freedom – The username and password authentication protect the library system from any unauthorized access. Since the users have only read access to the library database the database is secure.
The student and staff users must get the online library access username and password allocated from their respective departments. The library firewall access list is configured based on the inputs received from the department heads.
Recognition rather than recall – The user interface of the library automation system provides easy access to the information and the user can easily surf from one screen to another.
The drop-down menu on screen 1 provides the list of subjects taught by the department, the student can select a subject and the title box will show the list of book titles in the library on the given subject. After choosing the title the list of authors on a given subject title is available. A description of the subject, title and author is shown in a pop-up window. Users can scroll using keyboard keys, mouse, or touchpad.
Help and Documentation – The library automation system has help documentation written in simple English that is comprehensive and easily understandable. A ‘help’ button is present on every window and it presents a pop-up help description on the given window.
Interpretation
The PACT analysis of the library automation system highlighted the following strengths in the library automation system:
- The online remote login interface provided 24hr access to the library.
- Connectivity with the WorldCat database provides an opportunity to find the latest research material.
- The ILL facility makes new research material available for the students & staff of the institution.
- The interactive user interface was easily usable by nonIT students.
The following weaknesses were found in the library automation system:
There was no feature to inform students about the new arrivals in the library. The CIP information received by the library was communicated to the teaching staff. This information was investigated by the staff and if found appropriate for the institution course it was added to the syllabus and provided to the students. The students had to log in to the library automation system often to find if there are any new updates to the catalog.
The advancement in web technologies may make the SQL-based system obsolete in the future. The OCLC architecture of a centralized asset database requires that libraries also register their assets to the WorldCat. The library must start adapting to the new metadata description language (RDF topic maps) based system to implement new features based on ontology.
Suggested Improvements
After usability test and PACT analyses following suggestions are made for the improvement and enhancement of the library automation system:
Based on the context analysis it is suggested that if the library has sufficient assets to meet the requirements of the department students the extra resources can be put into a general pool that can be accessed without the specific department permission. Computer Science courses are popular with other course students and they can take advantage of the library assets without seeking computer science department permission.
Based on the technical analysis it is suggested that the library automation system be ported to RDF-based architecture to enable integration with the web 3.0 online systems. RDF metadata will provide one authorized information resource as desired by most students.
The library can start publishing a blog on the institution’s intranet for the benefit of the students and staff. The blog can elaborate on the advanced features of the tool as well as information about new updates.
The library can publish an RSS feed for every department/subject to inform about new CIP information. This will assist students to know about new research and arrivals.
Student volunteers can be assigned the job of converting library records information into RDF metadata.
Conclusion
The author has put extensive effort into the research of the library automation system usability test and PACT analysis. The result of the research is described in the observations section and suggested improvements have been listed. The author has conducted an online search about OCLC’s latest research into library automation systems and has found that users of the institution library automation system can be increased manyfold by the addition of new web 2.0 & 3.0 applications such as blog and RSS. The online interface of the library is a social networking feature that makes students aware of the courses, teaching material, and research conducted in other institutions.
The author has selected three usability principles out of the ten principles suggested by Jakob Nielsen. Jakob Nielsen is known as a ‘guru of Web page usability and is the principal of Nielsen Norman Group. The emphasis of this usability test has been the ease of use for a nonIT user. Can a nonIT user perform operations on this library system without fear and with confidence? The answer to this question based on the survey is YES. The survey was conducted for noncomputer science students since in the internet era most students in the institution are IT aware. The objective of the PACT analysis was to find if the latest technology is being used to provide the maximum benefit to the users. Suggestions have been made on how to use the RSS and blog to increase student productivity. It was found that online help and documentation reduces dependence on the library staff thus reducing library operational cost.
The author has found that the usability methods questionnaire survey and heuristic evaluation was very effective in the testing. A questionnaire survey is one of the best usability methods to get the opinion of other users on the user interface and utility of the system. Heuristic evaluation was used to get the first-hand performance data.
References
106CR Designing for Usability. [internet] Web.
Business benefits of usability. UPA. [internet] Web.
Guide to the Harvard System of Referencing.2007. University Library. Angila Ruskin University. [internet] Web.
Horizon Integrated Library System. SirsiDynix. [internet] Web.
ISO/IEC 25062. 2006. Software Engineering – Software product Quality Requirements and Evaluation (SquaRE) – Common Industry Format (CIF) for usability test reports.
Nielsen, Jakob. 2005. Ten Usability Heuristics. [internet] Web.
PACT Analysis. [internet] Web.
Products and Service. OCLC. [internet] Web.
RDF topicmaps. OCLC. [internet] Web.
Usability in Software Design. 2000. Microsoft Corporation. [internet] Web.