A Survey on Information Systems Interoperability

In this survey, some basic concepts are presented. It's pretty good to go over these concepts in this paper. Its structure is clear and it's easy to understand most of the content. The author had the rich experience to apply the techniques of information systems into the agriculture. I think that is a system to manage multiple database containing agricultural information, therefore it might be pretty helpful on our work, since our goal is to construct a data-integration system to manage multiple archeological database.

Now, let's take a look at his paper section by section, (the easiest way to review J)

1. In the first section, the motivation and the basic method used are discussed respectively. 
1. the goal is the construction of data warehouse (or materialized view) integrating several kinds of data sources, particularly for scientific applications in agriculture. Interesting, because it's a little similar to our goal in KADIS, the only difference is that our application is about archeology. 
2. The background of the application is that: distinct data sources may be maintained independently, and research on semantic data heterogeneity is focused, and an incremental and modularized approach is suggested by the paper to deal with the issues of data integration. 
2. Information System Interoperability
1. The paper suggests that the only way to reach interoperability is by publishing the interfaces, schemas and formats used for information exchange, making their semantics as explicit as possible, so that they can be properly handled by the cooperative systems. 
2. Three viewpoints must be considered about the information systems' interoperability: application domain, conceptual design and software system technology. For each viewpoint, interoperability should be achieved.
3. Data System Interoperability
1. in this section, the definitions of centralized database system and heterogeneous database system are firstly presented.
2. Two categories of approaches to enable integrated access to multiple physical databases: schema integration and the federated approach. 
3. web database is also mentioned in the section, and the challenge of the querying Web database research is the construction of a unified and simple interface.
4. Data Integration
1. The basic procedure of data integration concerned here include the resolution of heterogeneity conflicts and transformations of source data to accommodate them in the integrated view.
2. The kinds of data to be integrated and the heterogeneity conflicts should be firstly categorized. 
3. The structure of data is discussed: the structured data and semi-structured data. 
4. Data heterogeneity, or conflict is summarized. Two ways are used to define the data conflicts or data heterogeneity. 
1. representational conflict and semantic conflict;
2. based on the different levels of abstraction, such as instance, schema, data model. The conflicts can be classified as : data conflicts, schema conflicts, data versus schema conflicts, and data model conflicts. 
5. Some proposals are brought up to solve these conflicts in the RDB and semi-structured data. Here some surveys need reading. 
6. Another way to solve the conflict, is the construction of the standard to describe the semantics. (common semantics)
7. A series of procedures are suggested in the paper to generate the unified view of heterogeneous data. It's inspiring for our work, I think.
5. Building blocks to integrate data in cooperative systems
1. In the section, the author describes the software framework, modules, and techniques that could be used to contribute the integrated data views.
6. The semantic web
1. A simple digest is presented here, giving us a general idea of the semantic Web standards and technologies. 
1. Character Encoding + URI 
2. XML + Schema
3. RDF + RDFS
4. Ontology
7. Web services

Semantic Integration Research in the Database Community  A Brief Survey

This survey firstly presents the applications that require semantic integrations, and brings up difficulties during the integration process. Schema matching and data matching, which play very important influence in the semantic integration, are focused on in the following content. Finally, some other open issues are discussed. 

Applications mentioned in the survey include:

1. schema integration: merging a set of given schemas into a single global schema. The basic procedure is match - merge.
2. Translate data between multiple databases. Data coming from multiple sources must be transformed to data conforming to a single target schema.
3. Data integration system: provide the user with a uniform query interface (mediated schema) to a multitude of data sources. It's very inspiring, because our work on KADIS will belong to this category.
4. P2P, peer data management: allow peer to query and retrieve data directly from each other, without building the mediated schema.
5. Model management : create tools for easily manipulating models of data. 
6. Many factors increase the need for the applications above, such as the rapid development of the Internet, the widespread of adoption of XML as a standard syntax to share data, the growth of the Semantic Web etc. 

However, semantic integration is an extremely difficult problem, because of the challenges below:

1. It's hard to infer the semantics of the involved elements, unless the creators of data, or exports are available.
2. The clues used for matching schema elements are often unreliable. Sometimes, even the clues are not complete.
3. The global nature of matching makes it costly to find the BEST matching elements in different sources.
4. Depending upon the applications, the matching is often subjective. That means, the criteria of matching might changes with the changes of applications.

Next, the survey discuss the progress in schema matching and data matching respectively. Three aspects are concerned, they are:

1. Matching techniques:
1. Rule-based solutions: hand-crafted rules to match schemas (domain-independent, and domain-specific)
1. Benefits: inexpensive; fast because it typically operates on schemas; work very well in certain types of applications; capture valuable user knowledge about the domain.
2. Drawback: can not exploit data instances effectively; can not exploit the previous matching efforts to assist in the current matchings. 
2. Learning-based solutions: neural network, Naive Bayes etc. are used in this category; external evidence, like past matches, is exploited; domain-specific schemas and matches, even the users are involved in this solution.
2. Architecture of matching solutions: module-based multi-matcher architecture, each module exploits well a certain type of information to predict matches. 
1. On the other hand, domain-specific information is taken as global information to be handled after the matching phase. 
2. Related work in knowledge-intensive domains.
3. Types of semantic matches:
1. one-to-one match
2. complex match : need domain knowledge; the correct complex match is often not the top-ranked match, but somewhere in the top few matches predicted.

Data matching is discussed in the following: this part is important, because the main goal of KADIS is to match and query data from multiple sources. The techniques used in data matching are similar to those in schema matching, even though its focus is on the tuple matching. 

Finally, the paper also discussed the open issues related to the whole semantic integration process, beyond matching schema and data. These issues include: 

1. User interaction: the key point is how to reduce the user's burden during the user interaction with the system;
2. Formal foundations:  develop formal semantics of matching and try to formally explain the mechanism of matching.
3. Industrial strength schema matching: apply algorithms in real world setting. Help to understand better the applicability of current research and suggest future direction.
4. Mapping maintenance: dynamically maintain the semantic mappings between different sources, because of the changes of the source data.
5. Reasoning with imprecise matches on a large scale: use and evaluate the system where parts of the mapping always remain unverified and potentially incorrect because of the overwhelming size of the information.
6. Schema integration: construct a global schema through matching and merging schemas from multiple data sources, which is about the high-level operations, like model management etc. 
7. Data translation: elaborate matches into mappings, to enable the translations of queries and data across schemas;
8. Peer-to-Peer data management: 

Some questions about the Curve Segmentation

Last night, I finished the implementation of the curve segmentation algorithm in Java. I am thinking about some questions about the application of this algorithm in the topic segmentation. 

1. Now we used MDS to extract the most important feature in the inputting text stream, and some information is lost during this extraction. The amount of lost information depends upon the degree of the complexity of data. That is, much information could be lost, when MDS can not find the MOST important feature: no dominant feature exist at all, or not only one feature is important of all etc. 

Since the following procedure to do the curve segmentation is on the basis of the dominant features extracted through MDS, it's not clear that we could get the accurate segments about the information stream using the features that can not reflect the whole graph. 

Thus, the question is, how to keep the accuracy of representative features. One solution is to use a method rather than MDS to analyze the dominant features of the inputting information stream, the other is to use complex features as the representative of the data. 

MDS is kind of dimension reduction method, which tries to capture common features in the data, and compress the representative without losing much accuracy. In theory, it's not very effective to use other dimension reduction methods if MDS has done a pretty good work. Maybe we'd better try the second solution: adopt the curve segmentation to the high dimension space. 

2. Our goal in the project is to analyze the information stream, which is often dynamically changed with the time. However, the method we used to do the curve segmentation requires the whole graph. That's, we have to analyze the data from the scratch when some changes happen, in order to exploit MDS to reduce the dimensions of feature space. One consequence is that the patterns we got just . That means, the analysis result can change as more events occur in the stream. It sounds not good, although reasonable. Usually, people are easily lost in the changing world! Therefore, the question is: how to segment the curve in an incremental way with keeping the consistency of the result as much as possible. I don't know the answer now. One idea is to construct different segmentation based on the sizes of window containing information in the stream along the time line. 

Semantic Integration Research in the Database Community  A Brief Survey

This survey firstly presents the applications that require semantic integrations, and brings up difficulties during the integration process. Schema matching and data matching, which play very important influence in the semantic integration, are focused on in the following content. Finally, some other open issues are discussed. 

Applications mentioned in the survey include:

1. schema integration: merging a set of given schemas into a single global schema. The basic procedure is match - merge.
2. Translate data between multiple databases. Data coming from multiple sources must be transformed to data conforming to a single target schema.
3. Data integration system: provide the user with a uniform query interface (mediated schema) to a multitude of data sources. It's very inspiring, because our work on KADIS will belong to this category.
4. P2P, peer data management: allow peer to query and retrieve data directly from each other, without building the mediated schema.
5. Model management : create tools for easily manipulating models of data. 
6. Many factors increase the need for the applications above, such as the rapid development of the Internet, the widespread of adoption of XML as a standard syntax to share data, the growth of the Semantic Web etc. 

However, semantic integration is an extremely difficult problem, because of the challenges below:

1. It's hard to infer the semantics of the involved elements, unless the creators of data, or exports are available.
2. The clues used for matching schema elements are often unreliable. Sometimes, even the clues are not complete.
3. The global nature of matching makes it costly to find the BEST matching elements in different sources.
4. Depending upon the applications, the matching is often subjective. That means, the criteria of matching might changes with the changes of applications.

Next, the survey discuss the progress in schema matching and data matching respectively. Three aspects are concerned, they are:

1. Matching techniques:
1. Rule-based solutions: hand-crafted rules to match schemas (domain-independent, and domain-specific)
1. Benefits: inexpensive; fast because it typically operates on schemas; work very well in certain types of applications; capture valuable user knowledge about the domain.
2. Drawback: can not exploit data instances effectively; can not exploit the previous matching efforts to assist in the current matchings. 
2. Learning-based solutions: neural network, Naive Bayes etc. are used in this category; external evidence, like past matches, is exploited; domain-specific schemas and matches, even the users are involved in this solution.
2. Architecture of matching solutions: module-based multi-matcher architecture, each module exploits well a certain type of information to predict matches. 
1. On the other hand, domain-specific information is taken as global information to be handled after the matching phase. 
2. Related work in knowledge-intensive domains.
3. Types of semantic matches:
1. one-to-one match
2. complex match : need domain knowledge; the correct complex match is often not the top-ranked match, but somewhere in the top few matches predicted.

Data matching is discussed in the following: this part is important, because the main goal of KADIS is to match and query data from multiple sources. The techniques used in data matching are similar to those in schema matching, even though its focus is on the tuple matching.