Knowledge Engineering of Educational Systems for Authoring System Design

-- A preliminary results of task ontology design --

 

Riichiro Mizoguchi+, Katherine Sinitsa++, and Mitsuru Ikeda+

+ ISIR, Osaka University, Japan

++Glushkov Institute for Cybernetics, Kiev, Ukraine

miz@ei.sanken.osaka-u.ac.jp, ikeda@ei.sanken.osaka-u.ac.jp, kath@its.freenet.kiev.ua

 

Abstract: This paper presents a preliminary result of task ontology design of IES. Task ontology provides us with an effective methodology and vocabulary for both analyzing and synthesizing intelligent educational systems(IESs). It fills the gap between users and authoring systems. This paper discusses how task ontology contributes to characterizing intelligent educational systems and hence to facilitating authoring systems design with fluent communication capability with users. We begin our paper by discussing what an ontology is followed by discussion on its roles in IES research. Preliminary results of ontology design at an abstract level is shown together with some specifications of a counter example and its use. We demonstrate the utility of task ontology by specifying how to communicate with an authoring system.

 

 

1. INTRODUCTION

 

Viewing from knowledge engineering point of view, an authoring system should provide users with a sophisticated vocabulary supported by computational mechanisms which enables them to represent their ideas about educational systems they want to build at the right level of abstraction. While some of the existing authoring systems are nice as a software development environment, they are still weak at communicating with users and not very efficient in satisfying users' various requirements(Major, 1993)(Van Marcke et al., 1995)(Vivet, 1989). These are mainly caused by the lack of knowledge engineering of educational systems.

In order to design an authoring system, designers have to know what an educational system is, that is, what functions are necessary for what goal of education, what components are necessary for what functionality, how to specify each components, what architecture is appropriate for what types of education, how to control the behavior of the components, etc., since it should reflect such fundamental understanding of educational systems. Issues here include how much we know about these fundamental characteristics of them. There have been much philosophical discussion and implementation of educational systems together with evaluation. However, there is little in between the two. What we usually see are very abstract discussions and idiosyncratic implementations (Wenger, 1987). What we need for filling up this gap are well-designed common vocabulary and frameworks for educational systems. We also need to formalize educational tasks at the right level of abstraction. We need to explicitly define learner's and program's roles and tasks in educational processes. We are able to find a nice solution to this problem in knowledge engineering.

Knowledge engineering has been considered as technology of building expert systems. It has contributed to eliciting expertise, organizing it into a computational structure, and building knowledge bases. While rule base technology has dominated until recently, a new technology based on knowledge modeling has appeared such as KADS project in Europe(Wielinga, 1992), PROTEGE project in USA(Puerta et al., 1992), and MULTIS project in Japan(Mizoguchi et al., 1992)(Mizoguchi et al., 1995b). All these technologies are originated from the idea of Generic tasks(Chandra, 1986) and heuristic classification(Clancey, 1985). The latest knowledge engineering technology comes up with an idea of task ontology which serves as a theory of vocabulary/concepts used as building blocks for knowledge-based systems(Mizoguchi, 1993)(Mizoguchi et al., 1995a).We consider ontology consists of task ontology which characterizes the computational architecture of knowledge-based systems and domain ontology which characterizes the domain knowledge.

Task ontology provides us with an effective methodology and vocabulary for both analyzing and synthesizing knowledge based systems which intelligent educational systems(IESs) belong to. Ontology is not only a pure theory in fundamental AI. Rather, it is becoming a research field called "Ontology Engineering"(Mizoguchi, 1996a) like "Knowledge engineering" in expert systems. Ontology provides us with what we need to overcome the shortcomings the current IESs have we discussed above.

This paper discusses how task ontology contributes to characterizing intelligent educational systems and hence to facilitating authoring systems design with fluent communication capability with users. The next section describes what an ontology is. Section 3 discusses task ontology as well as its roles in IES research. In section 4 we discuss preliminary results of ontology design at an abstract level. Its details is found in (Mizoguchi et al., 1996b) and http://www.ei.sanken.osaka-u.ac.jp/announce/ITS.ws.html. We demonstrate the utility of task ontology by specifying how to communicate with an authoring system in section 5.

 

 

2. WHAT IS AN ONTOLOGY?

 

2.1 Simple definitions

 

Three simple definitions are given below.

 

  1. Ontology is a term in philosophy and its meaning is "Theory of existence".
  2. In AI, ontology is defined as "An explicit representation of conceptualization" (Gruber, 1992).
  3. In KB community, ontology is defined as "a system of primitive vocabulary/concepts used for building artificial systems"(Mizoguchi, 1993). Although these are compact, it is not sufficient for in-depth understanding what an ontology is. A more comprehensive definition is given in the next subsection.

2.2 Comprehensive definitions

 

Following Guarino(Guarino et al., 1995), we use the convention in which capital letter "O" is used to distinguish the "Ontology" in philosophy from others. "Ontology" is a theory which can answer questions such as "what is existence", "What properties exist common to all the existence", "what properties can explain the existence", "How these properties explain the existence", etc. On the other hand, "ontology" starting by small "o" represents our ontology whose design methodology is like one for Ontology, but the target is different from it. Not the "existence" but smaller and more concrete thing such as enterprise, thermo-dynamics, problem solving, tutoring, etc. are discussed. We define an ontology as an explicit and unambiguous description of concepts and relations among them appearing in the target thing. Such ontologies exist as many as the possible target things. We do not have to use logic to describe it.

Formal ontology is an axiomatic description of an ontology. It can answer questions on the capability of ontology. Axioms in a formal ontology has two roles as follow: 1) To represent the meaning of concepts rigorously. 2) Within the scope of the knowledge represented declaratively, to answer the questions on the capability of the ontology and things built using the concepts in the ontology.

Questions about the capability of ontology plays an important role in its evaluation and they are divided into the following two: 1) Questions on the formal properties of the ontology and things designed using ontology. 2) Questions on the behavior of the things designed using the ontology. The former is called "competence" question and the latter "performance" question. Axioms written in predicate calculus are sufficient for answering the former. To answer the latter question, however, we often need procedural engines to interpret the meaning of concepts in the ontology because declarative knowledge with a formal prover cannot answer all the questions. To cope with such situations, we introduce axiom equivalents defined as follows:

 

Axiom equivalent: An axiom equivalent is not a rigorous or declarative axiom based on formal inference engine, but it is partially declarative knowledge based also on interpretation by a procedural engine to answer performance questions. Axiom equivalents do not have to be formalized completely.

 

3. TASK ONTOLOGY

 

3.1 What is a task ontology?

 

Task ontology is a system/theory of vocabulary for describing inherent problem solving structure of all the existing tasks domain-independently. It is obtained by analyzing task structures of real world problems. Design of task ontology is done in order to overcome the shortcomings of generic tasks(Chandra, 1986) while preserving their basic philosophies. It does not cover the control structure but do components or primitives of unit inferences taking place during performing tasks. The ultimate goal of task ontology research includes to provide all the vocabulary necessary for building a model of human problem solving processes.

When we view a problem solving process based on search as a sentence of natural language, task ontology is a system of semantic vocabulary for representing meaning of the sentence. The determination of the abstraction level of task ontology requires a close consideration on granularity and generality. Representations of the two sentences of the same meaning in terms of task ontology should be the same. These observations suggest task ontology consists of the following four kinds of concepts:

 

 

  1. Generic nouns representing objects reflecting their roles appearing in the problem solving process,
  2. Generic verbs representing unit activities appearing in the problem solving process,
  3. Generic adjectives modifying the objects, and
  4. Other concepts specific to the task.

Terms/concepts in task ontology for scheduling tasks, for example, look as follows:

 

Nouns: Scheduling recipient, Scheduling resource, Due date, Schedule, Constraints, Goal, Priority, etc.
Verbs: Assign, Classify, Pick up, Select, Relax, Neglect, etc.
Adjectives: Unassigned, The last, Idle etc.
Others: Strong constraint, Constraint satisfaction, Constraint predicates, Attribute, etc.

Verbs are defined as a set of procedures representing its meaning. So, they collectively serve as a set of reusable components for building IESs.

 

3.2 Roles of task ontology in IES

 

Roles of task ontology include:

 

  1. To enable research results to accumulate(Mizoguchi et al., 1996a)
  2. To provide vocabulary/concepts in terms of which one can compare and assess existing IESs.
  3. To formalize educational tasks.
  4. To specify the tutoring/training context which contributes to making it easy to put domain knowledge into a right context, since it provides us with abstract roles of various objects which could be instantiated to domain-specific objects.
  5. To provide reusable components for IES design and development.
  6. To enable translation of the knowledge-level description of the problem solving process into symbol-level executable code.
  7. To standardize communication protocol among component agents of IES in CSCL(Ikeda et al., 1995).

4. PRELIMINARY DESIGN OF TASK ONTOLOGY OF IESs

 

We are currently designing task ontology for IESs and trying to represent it in an ontology description language. Because of the space limitation, we only present higher level description of task ontology designed thus far. Details of it is found in (Mizoguchi et al., 1996b) and http://www.ei.sanken.osaka-u.ac.jp/announce/ITS.ws.html.

The top-level categories of task ontology of IESs consist of Goals of education, Learner's state, System's functionality, Learner-system interaction, and Teaching material knowledge because an IES is characterized as an interaction between a system and a learner in which the system's activity is based on its functionality which is performed in a domain according to the learner's state under a certain goal. Then, we have the following top level categories of concepts.

 

4.1 Goals of education

 

Let us investigate Goals of education first. There have been proposed a number of paradigms for IESs to date. While they are seemingly conflicting each other, the reality is not. When we carefully investigate the paradigms, we easily understand most of the them can co-exist, since they have different educational goals. In this sense, goals enable one to distinguish and identify an appropriate paradigm for his purpose.

Goals of education are first divided into two categories such as augmentation of Domain-independent and Domain-dependent capabilities. The former is mainly related to Reasoning capability which has various kinds of subcapabilities. The latter is divided into three subcategories such as Deep understanding of concepts(declarative knowledge), Problem solving capability(procedural knowledge) and Skills.

 

4.2 Learner's state

 

Learner's state is composed of Phase in learning process, Knowledge state, and Mental state. Knowledge state consists of Numerical representation and Symbolic representation. Symbolic representation is composed of two components such as Location of bugs and Types of bugs.

 

4.3 System's functionality

 

Needless to say, system's functionality is the most important category in IESs. The top level concepts are concerning how to teach which characterizes the type of IESs. Assuming autonomous systems, it includes One-to-one interaction and Group learning. The former incudes Repetitive practice, Learning by doing, Free exploration in a learning environment, Interactive learning environment, Coaching, Tutoring, Training and the latter Collaboration, Coordination, Cooperation, Game-playing, Argumentation, etc.

We also identify non-autonomous systems, that is, tools used with the aid of a human teacher. In this paper, however, we only discuss tutoring systems as a typical autonomous system. Functionality of an ITS includes Modelling and Tutoring. The former characterizes capability of an ITS to model a learner as well as the problem domain which is critical to make an IES behave intelligently. Tutoring activity is composed of Tutoring objectives, Control, and Methods. Methods are composed of Actions and Objects. The former includes Help which simply gives information required upon request, Getting learners motivated which encourages and compliments learners, Exercise which gives learners problems to solve, Guide which gives Explanation or Hints appropriate for the learner's understanding state and context, and Evaluate/Assess. Objects includes Problems, Explanation, Hints, Advice, and Learner's performance.

 

4.4 Interaction between the system and the learner

 

Learner's activity, both internal (mental) and external (actions, communication) also seems to play an important role. However, as we cannot directly assess/evaluate his/her mental activity and student model construction is mainly consists of filling the chosen framework using learner-system communication as an information source, we will concentrate here on Learner-system interaction. At the top level we suggest to characterize interaction by the following categories: Mode of interaction, Communication roles, Content type and Control/Sequencing/Protocol. The first concerns technical means and recognition techniques incorporated in IES, Communication roles reflect learner's attitude to IES, Content types describe cognitive content of communication and Control - a communication order.

 

4.5 Teaching material knowledge

 

Teaching material is a heart of education. To make it easier to manipulate various teaching material, it is important to characterize it in terms of a few concepts. We analyzed various teaching material knowledge from the viewpoint of effectiveness of tutoring strategy applications to them. We first came up with the following:

 

 

  1. Ease of getting a guideline of error correction
  2. Ease of acquiring necessary attributes
  3. Applicability of verification operation
  4. Amount of resource consumption in the problem solving process

These are factors which explain why a certain set of tutoring strategies are not effective to a certain set of domain knowledge. And then, we came up with the following four attributes such as Declarative/procedural, Abstract/concrete, Formal operations on the topic are defined or not, and The size of search space for that purpose. Each attribute corresponds to a factor above one by one. Those who are interested in the detailed explanation of this characterization, please refer(Ikeda et al., 1994).

Teaching material knowledge includes Domain knowledge, Search control knowledge and Strategic knowledge. The former is composed of Nodes and Links. Nodes have several attributes such as mandatory/optional, difficulty to master, etc. and include Concepts, Facts, Rules, and Principles. Links includes Prerequisite, Objectives, is-a, part-of, order, etc. Search control knowledge includes Goals, Cost/Score, Preference, Focus, etc.

 

4.6 Some definition of concepts

 

We here pick up some typical verb, nouns, and adjectives appearing in our ontology together with informal definition.

 

Verbs:

Nouns:

Bug: A piece of incorrect/missing knowledge
Explanation: Explicit information which helps a learner get better understanding
Hint: Implicit information(a suggestion or stimulus) which helps a learner get better understanding
Examples: An instance of a class of object used in explanations/hints
Counter example: An example used in a hint, which is produced by supposedly the same (wrong) method, that was used by a learner, and which obviously violates pre-defined/some/known conditions.
..........

 

In addition to the above concepts, we need some concepts specifying objects, that is, adjectives as follows:

 

Generic Adjectives:

Current, Relevant, Confirmed(hypothesis), Observed/Known(facts), Unknown, Evaluated, Missing, Correct, Incorrect, Mastered, Next(topic), Easier, More-difficult, Analogous, Focussed, etc.

 

Constraint adjectives:

Satisfying, Maximal, Minimal, Larger, Smaller, Largest, smallest, Best, Better, etc,

 

We need more verbs for describing the derivation process or something implicit. Their roles in task ontology is not to realize the corresponding operation but to modify or specify objects just as adjectives.

 

General verbs:

Negate, Support, Derive, Solve, etc.

 

 

5. SPECIFICATION OF IESs USING ONTOLOGY

 

5.1 Formulation of a counter example

 

We here demonstrate how our ontology contributes to specifying the activity of a typical building process of an ITS. Let us take a situation where the system is going to present a learner a counter example(ce). We first define a predicate ce. Now, let px and py are problems of the same class with different parameters and x and y are solutions of them. And, s, l and m for a system, a learner and a method. Let a predicate ce(l,m,px,x,s,py,y) stands for answer y of problem py is a counter example of x of problem px for learner l who uses method m. Then, ce(L,M,Px,X,Py,Y) is defined as follows:

 

 

   ce(L,M,Px,X,Py,Y):-
       solve(L,Px,M,X),incorrect(X,Px),believe(L,correct(X,Px)),
       solve(s,Py,M,Y),incorrect(Y,Py),believe(L,incorrect(Y,Py)),

where solve(L,Px,M,X) reads answer X is obtained by solving problem Px by learner L using method M, incorrect(X,Px) answer X of Px is incorrect, believe(L,correct(X,Px)) learner L believes X is a correct answer of Px, and solve(s,Py,M,Y) answer Y is obtained by solving the problem Py by the system, s, using the same method M as that the learner used.

This is a formulation of ce. In the implementation of ce, M(method) should be modeled to know what method the learner used(how he/she solved the problem) and some necessary facts have to be stored in the learner model or a fact data base before hand. Now let us show a rough image of how to implement ce mechanism. Let the problem be to learn what a rectangle is. To allow the system to use ce as its reaction to the learner's wrong answer, the authoring system must be able to match learner's answer with the correct definition and to model learner's definition ("M" in ce), and to find such an instance among X that satisfies learner's definition, but does not satisfy the correct one and he/she knows it is not a rectangle as a fact. As you can see, the first step for this particular problem formulation could be implemented by preliminary definition of the objects. So, modeling could be kept as simple as possible.

In our case, the author must define:

 

Domain knowledge, containing the correct definition of a rectangle, say,

 

  rectangle(X):-four_edges(X), parallel_two_facing_edges(X),
                perpendicular_neighboring_edges(X).

Procedures which implement predicates including the three appearing in the correct definition. What they do is to analyzes the two-dimensional image of instances of figures and decide if each predicate holds or not. Or, all the instances of figures could be prepared in advance in fact data base by defining all their shapes and logical attributes as follows:

 

Fact DB

 

four_edges(x1).		              four_edges(x2).
parallel_two_facing_edges(x1).	      parallel_two_facing_edges(x2).
perpendicular_neighboring_edges(x1).
not_perpendicular_neighboring_edges(x2).

in which xi stand for identifiers of instances of figures.

 

Learner model includes known facts about domain knowledge and definition of a rectangle, which, the system thinks, is used for problem solving - in this case it can be a wrong definition.

 

 

    rectangle(X):-four_edges(X), parallel_two_facing_edges(X).
    not_rectangle(X):-diamond(X).
    diamond(x7).

Thus, x7 is found as a ce. Necessary condition for using a ce, is an ability to model a learner's definition and known instances which could be a candidate of ce.

 

5.2 Specification of the situation in which ce is used

 

In principle, the system can use ce at anytime when it succeeds in finding one according to the definition/formulation. We next consider how to specify such situations. We only here present informal specification. Our aim here is to clarify how ontology helps authors(teachers) specify and implement their ideas about tutoring.

 

1) Tutoring objective: This kind of hint giving activity should be done under appropriate tutoring goals of Correcting bugs and Making learners to recognize bugs they have.

2) Phase in the learning process: The phase where ce is appropriate is In-depth understanding of new concepts in declarative cases and Training to solve a problem by using a known schema in procedural cases.

3) Inductive hint: A ce is considered as a kind of Inductive hint, since it suggests the learner to think the difference between the two conflicting examples.

4) Suggest hint: The action carrying out the ce in practice is Suggest. This is organized under Guide, under Method, under Tutoring and under System's functionality.

5) Types of bugs: In declarative knowledge cases, ce is effective when types of bug is Lack or Insertion of predicates(factors).

6) Modelling: In order to use ce appropriately, the system first builds a learner's model to identify the bugs. According to the definition of ce, the model should be able to be Symbolic and Executable. While building the model, the system may perform all the subactivities under Modelling. In case of simple implementation, as indicated above, the user could prepare possible buggy definitions of each figures to make the modeling easier where the systems only has to choose an appropriate buggy model out of them.

7) Other information used is the Reliability of the learner model. If it is lower than a threshold, then another action is taken. Or, Learner's general capability is low, then the system may prefer Explanation to Hint giving of ce.

 

5.3 Specification of user's requirements

 

In this section we would like to demonstrate how suggested IES ontology can be used by a teacher-author to formulate his requirements to the educational system to be designed and to choose an appropriate authoring tool for this purpose. Assume that an educational system is intended for training children to solve linear equations. Children are supposed to have obtained an initial instruction and explanations as to problem solving schema. A tutoring goal of the IES is to help children master problem solving skills, which could be reached by revealing their mistakes in operations and assisting them in recognizing and correcting their bugs. IES should provide an individual task sequence and help. Additional motivation is not necessary.

Now let us describe this requirement in terms of IES ontology items. This IES is an Autonomous systemfor one-to-one interaction, namely, Tutoring/Training. The main Educational goal is to train Domain-dependent capability, Problem-solving, Using a known schema. Learner-System Interaction is based on Learner-Teacher roles distribution and is System-driven, and Mode of interaction is Menu and Formal language for entering equations. Content of interaction includes Problems, Evaluation of Solution, Hint, Correct solution, Evaluation and Rule - from system and Solution from a learner via ask for Help.

Next, we define what kind of Functionality is necessary for meeting the author's requirements. Tutoring Objectives include Making learners to recognize bugs and Correcting bugs by Direct or Indirect instruction depending on learner's Degree of mastery. System should carry out a Dynamic problem selection based on Type of bugs, Degree of mastery and taking into account (Easier - More difficult - Similar) relations between Problems, Assesses each Solution process and Present a learner Evaluation results either for a step or a whole problem.

Methods to be incorporated in the IES include Exercises by Selecting/Generating and Presenting Problems; Evaluate Solution process and define probable Bug; Present Results of Evaluation and Suggest a Hint in a form of a Kind of bug, a Counter example or Specific example; Explanation of the Reason, Why the learner's Answer is incorrect; Help by Presenting Correct Answer and Prerequisite knowledge.

According to author's requirements, Learner model should represent his/her Bugs, their Types, Degree of mastery and History of problems given to avoid duplication, therefore, the modelling method employed must identify learner's knowledge state precisely enough. Typical Buggy model could be suggested for this purpose.

Tutoring strategies are also specified in terms of our ontology. Assume they are written in productions rules. Then, the following set of rules can be written:

 

If Reliability of the Learner model is lower than a threshold and the Answer is Incorrect

Then Present him/her it is Incorrect.

 

If Reliability of the Learner model is higher than a threshold, the Answer is Incorrect,

Tutoring objective is Correcting bugs, and the Type of bug is Lack of predicate

Then Suggest a Counter example and Suggest the Predicate is Missing.

 

The latter is a variant of the Rule 7 in (Major, 1993). More comprehensive representation of tutoring strategies adopted in WEST and GUIDON is found in (Mizoguchi et al., 1996b).

 

6. CONCLUSION

 

We have discussed ontology for IESs. Although the step we have made is not large, its implications to the future research in IESs area is not small. Because the research is in an initial stage, the ontology designed thus far is not very rich which should be augmented further. And, most of the description in this paper is informal. Although informal description is easy for humans to understand, they are sometimes ambiguous and computer cannot understand them at all. In order to enable computers manipulate ontology, it has to be formalized. Ontology helps identify a set objects that must be somehow defined, represented and implemented in authoring systems. Authors can refer to it in certain tutorial situations (which themselves could be specified by ontology), so the impact is - an identification of essential tutorial objects and their rigorous definitions which contribute to explication of fundamental conceptual structure of authoring systems and to filling the gap between users and authoring systems. The designers can think how to implement, and author - when and how to use. Furthermore, formal ontology implemented in a computer language enables us to build a support environment for use of authoring systems, IES evaluation and comparison, etc. Some examples of formalized ontology is found in (Mizoguchi et al., 1996b). Ontology for communication protocol between IESs for CSCL is not discussed in this paper, though it is also important. A preliminary result is found in (Ikeda et al., 1995). Ontology represented in a well-defined language with declarative and procedural semantics will contribute to formalization of IESs and hence to promotion of IES research in general.

 

REFERENCES

 

Chandrasekaran, B. (1986) Generic tasks for knowledge-based reasoning: the right level of abstraction for knowledge acquisition, IEEE Expert, 1, pp.23-30.

Clancey, W.J., Heuristic classification (1985) Artificial Intelligence, 27, 3, pp.289-350.

Gruber,T. (1992) A translation approach to portable ontology specifications, Proc. of JKAW'92, pp. 89-108.

Guarino, N. and P. Giaretta (1995) Ontologies and knowledge bases towards a terminological clarification, Proc. of KB&KS'95, pp.25-32.

Ikeda, M and R. Mizoguchi: FITS (1994) A framework for ITS -- A computational model of tutoring, J. of AI in Education, Vol.5, No.3, pp.319-348.

Ikeda, M and R. Mizoguchi (1995) Ontological issues of CSCL systems design, Proc. of AI-ED95, pp.242-249.

Major, N (1993) Reconstructing teaching strategies with COCA, Proc. of AIED'93, Edinburgh, pp. 66-73.

Mizoguchi, R. et al. (1992) Task ontology and its use in a task analysis interview system -- Two-level mediating representation in MULTIS --, Proc. of the JKAW'92, pp.185-198.

Mizoguchi, R.(1993) Knowledge acquisition and ontology, Proc. of the KB&KS'93, Tokyo, pp.121-128.

Mizoguchi, R, M. Ikeda, K. Seta, et al. (1995a) Ontology for modeling the world from problem solving perspectives, Proc. of IJCAI Workshop on Basic Ontological Issues in Knowledge Sharing, Montreal.

Mizoguchi, R., Y. Tijerino, M. Ikeda (1995b) Task analysis interview based on task ontology, J. of Expert Systems with Applications, Vol.9, No. 1, pp.15-25.

Mizoguchi,R, and M. Ikeda (1996a) Towards ontology engineering, Technical Report AI-TR-96-1, ISIR, Osaka University.

Mizoguchi, R. and K. Sinitsa, and M. Ikeda (1996b) Task ontology design for intelligent educational/training systems, Workshop on Architectures and methods for Designing Cost-effective and Reusable ITSs, ITS96, Montreal. http://www.ei.sanken.osaka-u.ac.jp/announce/ITS.ws.html

Puerta, A.R, et al.(1992) A multiple-method knowledge-acquisition shell for the automatic generation of knowledge acquisition tools, Knowledge Acquisition, 4, pp.171-196.

Van Marcke, K. et al. (1995) Learner adaptivity in generic instructional strategies, Proc. of AIED95, pp.323-333.

Vivet, M. (1989) Knowledge-based tutors -- Towards the design of a shell, International Journal of Educational Research, 12, pp.839-850.

Wenger, E. (1987) Artificial intelligence and tutoring systems, Morgan Kaufmann Publishers, California.

Wielinga, B.J. (1992) KADS: A modeling approach to knowledge engineering, J. of Knowledge Acquisition, Vol.4, No.1, pp.5-53.