Transcript Разработка программного обеспечения из

Tula State University
Faculty of Cybernetics
Laboratory of Information Systems
M. Bogatyrev, V. Tuhtin
Solving Some Text Mining
Problems with Conceptual
Graphs
2008
The Nature of Text Mining
Data mining:
"the nontrivial extraction of implicit,
previously unknown, and
potentially useful information from data"[1]
"the science of extracting useful
information from large data sets or databases."[2]
Text mining is interdisciplinary:
Text mining:
• process of deriving
high quality information from text;
• text data mining;
• text analytics
1.
2.
• information retrieval,
• machine learning,
• statistics,
Computational Linguistics
W. Frawley and G. Piatetsky-Shapiro and C. Matheus (Fall 1992), Knowledge Discovery in Databases:
An Overview, AI Magazine: pp. 213–228
D. Hand, H. Mannila, P. Smyth (2001). Principles of Data Mining. MIT Press, Cambridge, MA.
Computational (Corpora)
Linguistics
Text Mining
Knowledge Discovery
• text categorization,
• text clustering,
• concept/entity extraction,
• sentiment analysis,
• document summarization
• clusters,
• trends,
• associations,
• deviations
Global Problems
Problems
Processing objects
Natural Language Processing
Analysis of:
• syntax
• grammar
• morphology
• semantics
•annotation
•abstraction
• ontologies
•semantic roles
•Objects of tagging
Knowledge Models:
• rules;
• ontologies
Metadata
Corpora:
Data:
Plain text
• large and structured text
• tagging
Conceptual Graph:
Example:
“John is going to Boston by bus”
Concepts
Conceptual relations
Representations.
Conceptual Graph Standard by J. Sowa [8]
1. Conceptual Graph Interchange Form (CGIF)
[City*a:'Boston'] [Bus*b:''] [Person*c:'John'] [Going*d:''] (agent?d?c) (dest?d?a) (instrument?d?b)
2. XML Form
<graph id="35979486054" owner="0"> <type> <label>Proposition</label> </type>
<layout> <rectangle x="0.0" y="0.0" width="1500.0" height="1500.0"/> <color
foreground="0,0,175" background="0,0,175"/> </layout> … </layout> </arrow> </graph>
</conceptualgraph>
Applying Predicate Calculus
(CGIF + NOTIO)
(x : Go)(y : Person)(z : City)(w : Bus)( Name( y,' John ') 
 Name( z,' Boston ')  Agnt ( x, y)  Dest ( x, z )  Inst ( x, w))
Conceptual Graphs in
Digital Libraries
Supporting CGs in Digital Libraries:
1. Building and storing CGs
• Automated building of CGs
• Organizing access to CGs in Datastore
2. Solving applied problems with CGs
• Automated building and developing catalogues and
rubricators of DLs
• KDD problems
Supporting Conceptual Graphs :
Building and storing CGs
Standard way of building CG
Lexical restrictions are
needed:
•The sentences are marked with part-ofspeech tags.
• Some titles and sentences from
abstracts are filtered
•The selected sentences are parsed,
obtaining their syntactic tree.
• The syntactic tree is traversed and the
canonical conceptual graphs related to it
nodes are joined.
http://framenet.icsi.berkeley.edu/
http://wordnet.princeton.edu/
1. DL contains
scientific papers
2. Only abstracts are
transformed to CGs
Semantic Role
Labelling helps to
create conceptual
relations in CGs
Semantic Role
Labelling for CGs Building
“The working of a genetic algorithm is usually explained by the search
for superior building blocks”.
“John is going to Boston by bus”
http://l2r.cs.uiuc.edu/~cogcomp/srl-demo.php
Conceptual Graphs in Some Text Mining
Problems
1. Building Association Rules
R  {r1 ,..., rn }
T  {t1,..., tm}
-subsets represented
in T
{gˆ1 ,..., gˆ m }
gˆi  gk
- Association Rule
- Set of CGs
Generalization for concepts
Disjoin for relations
- transactional set
X  R, Y  R \ X
X Y
G  {g1,..., gn }
- initial set
- Set of generalized CGs
- Association Rule on CGs
m
t [X Y ]
j 1
Supported by
j
m
m
t [X Y ]
Having Confidence as
j
j 1
m
t [X ]
j 1
j


N [ gˆ , g ]

N [ gˆ ]
N [ gˆ , g ]

n
Conceptual Graphs in Some Text Mining
Problems
2. Building Ontologies by Aggregation of CGs
Supporting Contexts:
- with CGs:
- with Corpora:
In analyzing the ambiguities,
Wittgenstein developed his theory of
language games, which allow words to
have different senses in different
contexts, applications, or modes of use.
Solving Text Mining problems
by CGs clustering
CGs Hierarchy
• CGs Contexts problem
• CGs Similarity problem
?
Clustering algorithm for specific similarity measures
Conceptual Graphs Clustering
Similarity Measures
Conceptual similarity
Relational similarity
sc 
sr 
2n(Gc )
n(G1 )  n(G2 )
2m(Gc )
mGc (G1 )  mGc (G2 )
Some modifications of similarity measures
2n(Gc )l
sc 
,
n(G1 )  n(G2 )
 n(G1 )
k n  G  , if n(G1 )  n(G2 )

2
l
 k n(G2 ) , if n(G )  n(G )
1
2
 n(G1 )
mGc (G)  mboth  b1  b2  ...  bi
i  1,...,m  mboth
Unified similarity measure
s  d1sc  d2 sr
Genetic algorithm:
speciality of decisions
Ackley test function
y( x1 , x2 )  (20  e  20(e
0.5 ( x12  x22 )
 e0.5(cos2 x1cos2 x2 ) ))  30
Fitness function trajectories
Initial population
Final population
30
30
20Пригодность
10
0
20
20
2000
10
1800
0
1600
20
1400
20
1200
0
20
1000
0
Число поколений
0
50
-20
-20
100
150
200
250
300
0
-20
-20
Genetic algorithm for clustering
GA chromosomes representing the clustering for various encoding
schemes for clustering[5]:
(a) group number; (b) matrix; (c) permutation with the separator character 7;
(d) greedy permutation; (e) order based.
Clusters:{{
{XX
1, X
, 3X, X,6},
X {X
},{2, XX4,,XX5}
1
Our encoding scheme:
a1
a2
…
3
ai
6
…
2
4
, X 5}}
an
n objects
ai
picks the number of object which is in the same cluster as i -th object
Genetic algorithm for clustering
Chain encoding for Conceptual Graphs:
• realizes implicit parallelism of genetic algorithms;
•forces clustering algorithm to work faster;
• is invariant under similarity measure on CGs
a1
An idea about a possibility to vary
fitness function of GA by varying
its parameters
a2
…
ai
n objects
…
an
13
EVO – LIB Project
System’s architecture
Семантическая информация
Ассоциативные
связи
Модуль
выделения
ассоциативных
связей
Характеристические запросы
Объекты
Отношения
Пользователь
Набор
объектов
Ядро системы
Объекты
Структура БД
Агент
Отношения
Модуль оценки
индивидуумов
Модуль ГА
Характеристические
запросы
Структура БД
Метабаза
данных
Индивидуумы
Пригодность
Запросы
Модуль
агрегирования
Запросы
Результаты
Структура БД
Результаты
БД
Conceptual Graphs Clustering:
Data Example for Clustering
1.
We assume that the modality (i.e., number of local optima) of a fitness landscape is related to the
difficulty of finding the best point on that landscape by evolutionary computation (e.g., hillclimbers
and genetic algorithms (GAs)).
2. We first examine the limits of modality by constructing a unimodal function and a maximally multimodal
function.
3. At such extremes our intuition breaks down.
4. A fitness landscape consisting entirely of a single hill leading to the global optimum proves to be hard
for hillclimbers but apparently easy for GAs.
5. A provably maximally multimodal function, in which half the points in the search space are local optima,
can be easy for both hillclimbers and GAs.
6. Exploring the more realistic intermediate range between the extremes of modality, we construct local
optima with varying degrees of “attraction” to our evolutionary algorithms.
7. Most work on optima and their basins of attraction has focused on hills and hillclimbers, while some
research has explored attraction for the GA's crossover operator.
8. We extend the latter results by defining and implementing maximal partial deception in problems with k
arbitrarily placed global optima.
9. This allows us to create functions with multiple local optima attractive to crossover.
10. The resulting maximally deceptive function has several local optima, in addition to the global optima,
each with various size basins of attraction for hillclimbers as well as attraction for GA crossover.
11. This minimum distance function seems to be a powerful new tool for generalizing deception and
relating hillclimbers (and Hamming space) to GAs and crossover.
12. This paper describes an initial version of a library of sharable and reusable medical
ontological theories, organized according to a proposed classification of ontologies.
Conceptual Graphs Clustering:
clustering results
- applying conceptual nearness
- applying relational nearness
Resume
1.
2.
3.
4.
5.
Conceptual graphs is the perspective tool for
modelling semantics of texts in DL.
A process of creating ontologies can be based on
technologies which use conceptual graphs.
Conceptual graphs clustering helps in solving
structural problems in DLs and in understanding its
data.
Evolutionary approach is perspective in semantic
modelling with conceptual graphs.
To progress CGs technologies, a joined efforts of
computer specialists and linguists are needed.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
References
A World of Conceptual Graphs: http://conceptualgraphs.org/
Boytcheva, S. Dobrev, P. Angelova, G.CGExtract: Towards Extraction of Conceptual Graphs from
Controlled English. Lecture Notes in Computer Science № 2120, Springer 2001.
F. Southey J. G. Linders. Notio - A Java API for Developing CG Tools. 7th International
Conference on Conceptual Structures, 1999. P.p. 262-271.
Hirst G. Ontology and the Lexicon. - Handbook on Ontologies in Information Systems, Berlin –
Springer, 2003.
Cole, R. M. Clustering With Genetic Algorithms
http://citeseer.ist.psu.edu/cole98clustering.html.
Montes-y-Gomez, Gelbukh, Lopez-Lopez, Baeza-Yates, Text Mining at Detail Level Using
Conceptual Graphs. Lecture Notes in Computer Science Vol. 2393. Springer-Verlag, 2002. Pp.
122 - 136
Sarbo, J. Formal conceptual structure in language. In Dubois, D. M., editor, Proceedings of
Computing Anticipatory Systems (CASYS'98), pp. 289 - 300, Woodbury, New York. 1999.
Sowa R., Conceptual Graphs: Draft Proposed American National Standard, International
Conference on Conceptual Structures ICCS-99, Lecture Notes in Artificial Intelligence 1640,
Springer 1999.
Богатырёв М.Ю. , Латов В.Е. Исследование генетических алгоритмов кластеризации. - Изв.
ТулГУ. Сер. Математика. Механика. Информатика. Том 8, вып. 3 . Информатика. - Тула,
2002. - С. 101- 107.
Holland J.H. Adaptation in Natural and Artificial Systems, Ann Arbor: The University of Michigan
Press. Reprinted by MIT, 1992.
Растригин Л.А. Адаптация сложных систем. Рига: Зинатне, 1981. 375 с.
Емельянов В.В., Курейчик В.В., Курейчик В.М. Теория и практика эволюционного
моделирования. – М.: Физматлит, 2003 - 432 с.
Богатырёв М.Ю. Генетические алгоритмы: принципы работы, моделирование,
применение. Тула, ТулГУ, 2003. 152 с.
M. Bogatyrev. Modelling Systems With Symmetry// Proceedings of the 4 th International IMACS
Symposium of Mathematical Modelling. - Vienna, Austria, February 5-7, 2003.- ARGESIM-Verlag,
Vienna, 2003. - pp. 270 - 275.
M. Bogatyrev, V. Latov, K. Avdeev. Symmetry Based Decomposition and its Application in
Evolutionary Modelling. – Applied Mathematica: Proc. of 8 th International Mathematica
Symposium. Avignon, 19-23 June, France, 2006