Knowledge-based systems are increasingly found in a wide variety of settings and this handbook has been written to meet a specific need in their widening use. While there have been many successful applications of knowledge-based systems, some applications have failed because they never received the corrective feedback that evaluation provides for keeping development focused on the users' needs in their actual working environment. This handbook provides a conceptual framework and compendium of methods for performing evaluations of knowledge-based systems during their development. Its focus is on the users' and subject matter experts' evaluation of the usefulness of the system, and not on the developers' testing of the adequacy of the programming code. The handbook permits evaluators to systematically answer the following kinds of questions: Does the knowledge-based system meet the users' task requirements? Is the system easy to use? Is the knowledge base logically consistent? Does it meet the required level of expertise? Does the system improve performance? The authors have produced a handbook that will serve two audiences: a tool that can be used to create knowledge-based systems (practitioners, developers, and evaluators) and a framework that will stimulate more research in the area (academic researchers and students). To accomplish this, the handbook is built around a conceptual framework that integrates the different types of evaluations into the system of development process. The kinds of questions that can be answered, and the methods available for answering them, will change throughout the system development life cycle. And throughout this process, one needs to know what can be done, and what can't. It is this dichotomy that addresses needs in both the practitioner and academic research audiences.

The Handbook of Evaluation Methods for Health Informatics provides a complete compendium of methods for evaluation of IT-based systems and solutions within healthcare. Emphasis is entirely on assessment of the IT-system within its organizational environment. The author provides a coherent and complete assessment of methods addressing interactions with and effects of technology at the organizational, psychological, and social levels. It offers an explanation of the terminology and theoretical foundations underlying the methodological analysis presented here. The author carefully guides the reader through the process of identifying relevant methods corresponding to specific information needs and conditions for carrying out the evaluation study. The Handbook takes a critical view by focusing on assumptions for application, tacit built-in perspectives of the methods as well as their perils and pitfalls. Collects a number of evaluation methods of medical informatics Addresses metrics and measures Includes an extensive list of anotated references, case studies, and a list of useful Web sites

During recent decades we have witnessed not only the introduction of automation into the work environment but we have also seen a dramatic change in how automation has influenced the conditions of work. While some 30 years ago the addition of a computer was considered only for routine and boring tasks in support of humans, the balance has dramatically shifted to the computer being able to perform almost any task the human is willing to delegate. The very fast pace of change in processor and information technology has been the main driving force behind this development. Advances in automation and especially Artificial Intelligence (AI) have enabled the formation of a rather unique team with human and electronic members. The team is still supervised by the human with the machine as a subordinate associate or assistant, sharing responsibility, authority and autonomy over many tasks. The requirement for teaming human and machine in a highly dynamic and unpredictable task environment has led to impressive achievements in many supporting technologies. These include methods for system analysis, design and engineering and in particular for information processing, for cognitive and complex knowledge [1] engineering .

his volume of the series Lecture Notes in Medical Informatics contains the T proceedings of the Workshop on System Engineering in Medicine, which was held in Maastricht, The Netherlands, 16-18 March 1989. This workshop was sponsored by the EC under the framework of the Medical and Health Research Programme. The aim of the workshop was to assess whether there was sufficient support in the Medical Informatics community in the EC to establish a concerted action. This proceedings contain papers of the presentations given at the workshop. These presentations were centred around three themes: • Methods and Tools • Applications in the domains of chronic care and critical care • Evaluation of decision support systems The papers were prepared after the workshop and therefore we were able to include the relevant parts of the discussions which were related to the presentations. As a result of the discussions during the workshop, a proposal was prepared for the establishment of a concerted action, specifically addressing the development of guidelines for the evaluation of medical decision aids. This proposal was granted early 1990 under the same Medical and Health Research programme of the EC. Over 40 institutes are participating in this concerted action. It have been the outstanding presentations and the open discussions at the workshop that have been the starting point of this concerted action. The papers in this proceedings formed a starting point for the discussions in the meetings of the concerted action.

This book constitutes the refereed proceedings of the Second International Workshop on Autonomous Intelligent Systems: Agents and Data Mining, AIS-ADM 2007, held in St. Petersburg, Russia in June 2007. The 17 revised full papers and six revised short papers presented together with four invited lectures cover agent and data mining, agent competition and data mining, as well as text mining, semantic Web, and agents.