l8 - Text Summarization

Download Report

Transcript l8 - Text Summarization

1 SI 654

Database Application Design

Winter 2003 Dragomir R. Radev © 2002 by Prentice Hall

Database Processing

Eighth Edition

Chapter 12 Managing Databases with Oracle

2 David M. Kroenke © 2002 by Prentice Hall

What is Oracle?

3 • Oracle is the world’s most popular DBMS that… – Is extremely powerful and robust – Runs on many different operating systems – Can be configured and tailored – Operates with most, if not all, add on products © 2002 by Prentice Hall

Oracle Complexity

4 • The power and flexibility of Oracle makes it very complex: – Installations are difficult – The configuration options are numerous – System requirements are high – System maintenance is complex © 2002 by Prentice Hall

5

The Language of Oracle… SQL Plus

• SQL Plus triggers is used in Oracle to: – Define the structure of a database and the definition of the data – Insert, delete, and modify data – Define the behavior of the system through stored procedures and – Retrieve data and generate reports © 2002 by Prentice Hall

Gaining Access to SQL Plus

6 • To gain access to SQL Plus, you will need a username and password and possibly a host string (depending on your system configuration) • When Oracle is first installed, it establishes several default accounts, namely...

– internal/oracle (a privileged account) – sys/change_on_install (a privileged account) – system/manager (a privileged account) – scott/tiger (a non-privileged account) © 2002 by Prentice Hall

Creating the Database

7 • Ways to create an Oracle database: – Using SQL Plus • Start button –> Programs –> Oracle – OraHome81 –> Applications Development –> SQL Plus – Using Oracle’s Database Configuration

Assistant

• Start button –> Programs –> Oracle – OraHome81 –> Database Administration –> Database Configuration Assistant © 2002 by Prentice Hall

8

Entering SQL Plus Commands

• The SQL Plus Buffer – As a user types commands, the commands are saved into the SQL Plus buffer.

• The SQL Plus Editor – Users may edit or alter SQL Plus commands using a text editor.

© 2002 by Prentice Hall

SQL Plus Buffer Commands

9 • SQL Plus is not case sensitive (except within quotation marks). • List – displays the content of the SQL Plus buffer • List n • Change – display line number n and changes the current line number to n – performs a search/replace operation for the current line number • Semi-colon (;) or

slash

(/) executes © 2002 by Prentice Hall

SQL Plus Editor

• The SQL Plus Edit command will launch the SQL Plus text editor • After the SQL statement is complete and correct, exit the editor • To execute the statement, type the slash key (/) at the SQL prompt • To retrieve an existing SQL file: –SQL> Edit file1.sql

10 © 2002 by Prentice Hall

SQL Plus Commands

• Desc – lists the fields in the specified table • Select – retrieve data • Create – create objects • Drop • Alter – delete objects – change objects • Insert • Delete • Update – input data – delete data – change data 11 © 2002 by Prentice Hall

12

Select Syntax

Select field1, field2 From table_a, table_b; © 2002 by Prentice Hall

Create Syntax

Create Table tablename ( field1 data_type(size) NOT NULL, field2 data_type (size) NULL); Create Sequence tableID Increment by 1 start with 1000; (this command creates a counter that automatically increments for each new record – does not ensure uniqueness) 13 © 2002 by Prentice Hall

14

Alter Table Syntax

Alter Table tablename1 Add Constraint FieldPK Primary Key (Field1, Field2); Alter Table tablename2 Add Constraint FieldFK Foreign Key (Field1, Field2) references tablename1 On Delete Cascade; © 2002 by Prentice Hall

15

Insert Syntax

Insert into tablename (fieldID, field2) Values (fieldID.NextVal, ‘data content’); © 2002 by Prentice Hall

16

Drop Syntax

• Drop Table tablename; • Drop Sequence fieldID; © 2002 by Prentice Hall

17

Indexes

• Indexes are used to enforce uniqueness and to enable fast retrieval of data.

• Create Unique Index fieldIndex on Table(field1, field2); © 2002 by Prentice Hall

18

Changing the Table Structures

• Alter Table tablename add field4 datatype (size); • Alter Table tablename Drop Column field2; – you will permanently lose the data in field2 • Alter Table tablename Modify field3 not null; © 2002 by Prentice Hall

19

Changing the Data… Update Syntax

Update tablename Set field1 = ‘value_a’ Where field3 = value; © 2002 by Prentice Hall

Check Constraint

Provide a list of valid values or a valid range… Create Table tablename ( Field1 datatype (size) Not Null, Field2 datatype (size) Null Check (field2 in (‘value_a’, ‘value_b’))); 20 © 2002 by Prentice Hall

Check Constraint

• Alter Table tablename Add Constraint DateChk Check (DateField1 <= DateField2); • Alter Table tablename Add Constraint NumRange Check (Field1 Between 180 and 400); • Alter Table tablename Drop Constraint constraintname; 21 © 2002 by Prentice Hall

Views

• Displaying the data from the database just the way a user wants it… Create View View1 As Select * From Tablename With Read Only; 22 © 2002 by Prentice Hall

PL/SQL

• Allowing SQL to act more like a programming language. • Row-at-a-time versus set-at-a-time.

• PL/SQL permits Cursors • A

stored procedure

is a PL/SQL (or other program) stored in the database. A stored procedure may have parameters.

23 © 2002 by Prentice Hall

24

PL/SQL Parameter Types

• IN – specifies the input parameters • OUT – specifies the output parameters • IN OUT – a parameter that may be an input or an output © 2002 by Prentice Hall

PL/SQL Code

• Variables are declared following the AS keyword • The assignment operator is := as follows variable1 := ‘value’ • Comments in PL/SQL are enclosed between /* and */ as follows… /* This is a comment */ 25 © 2002 by Prentice Hall

26

PL/SQL Control Structures

FOR variable IN list_of_values LOOP Instructions END LOOP; IF condition THEN BEGIN Instructions END; © 2002 by Prentice Hall

Saving, Compiling, and Executing PL/SQL Code

• The last line in the PL/SQL procedure should be a slash (/). • The procedure must be saved to a file • To compile the procedure, type the keyword Start, followed by the procedure filename – START MyProg.SQL • To see any reported errors, type SHOW ERRORS; • To execute the procedure type EXEC MyProg (‘parameter1’, ‘parameter2’); 27 © 2002 by Prentice Hall

Triggers

• A

trigger

is a stored procedure that is automatic invoked by Oracle when a specified activity occurs • A trigger is defined relative to the activity which invoked the trigger – BEFORE prior to the activity – AFTER – execute the stored procedure – execute the stored procedure after the activity – INSTEAD OF – execute the stored procedure in lue of the activity 28 © 2002 by Prentice Hall

Trigger Example

Create or Replace Trigger triggername Before Insert or Update of fieldname on tablename For Each Row Begin /* instructions */ End; 29 © 2002 by Prentice Hall

30

A Trigger Knows the Old and New Values for Fields

• The variable :new.fieldname1 stores the new information for fieldname1 as entered by the user.

• The variable :old.fieldname1 stores the information in fieldname1 prior to the user’s request.

© 2002 by Prentice Hall

Activating a Trigger

• The trigger must be saved to a file • To compile the trigger, type the keyword Start, followed by the trigger filename – START MyTrigger.SQL • To see any reported errors, type SHOW ERRORS; • If no errors were encountered, the trigger is automatically activated 31 © 2002 by Prentice Hall

Data Dictionary

• The data dictionary contains information that Oracle knows about itself… the

metadata .

• It includes information regarding just about everything in the database including the structure and definition of tables, sequences, triggers, indexes, views, stored procedures, etc.

• The data dictionary table names are stored in the DICT table.

32 © 2002 by Prentice Hall

Concurrency Control

• Since Oracle only reads committed changes, dirty reads and lost updates are avoided • Transaction isolation levels: – Read Committed – Serializable – Read-only – Explicit Locks 33 © 2002 by Prentice Hall

34

Read Committed Transaction Isolation

• Reads may not be repeatable (2 reads may result in 2 data values, based on timing of updates and reads) • Phantoms are possible (data from a read may be deleted after the read occurred) • Uses exclusive locks • Deadlocks are possible and are resolved by rolling-back one of the transactions © 2002 by Prentice Hall

35

Serializable Transaction Isolation

• Reads are always repeatable • Phantoms are avoided • Must issue the following command: Set Transaction Isolation Level Serializable; or Alter Sessions Set Isolation Level Serializable; • Coordinates activities in submission order. When this coordination detects difficulties, the application program(s) must intervene. © 2002 by Prentice Hall

36

Read-only Transaction Isolation

• An Oracle-only isolation level • No inserting, updating, or deleting is permitted © 2002 by Prentice Hall

37

Explicit Locking

• Not recommended • Oracle does not promote locks. As a result, a table may have many, many locks within it. Oracle manages these locks transparently. Issuing explicit locks may interfere with these transparent locks.

© 2002 by Prentice Hall

38

Oracle Security

• Username and Password is used to manage DBMS access • Users may be assigned to one or more profiles • Oracle provides extensive resource limitations and access rights. These restrictions may be applied to users or profiles.

• The SQL Grant operator provides additional access rights • The SQL Revoke operator remove access rights © 2002 by Prentice Hall

39

Backup/Recovery

• Committed changes are saved to destination Tablespaces.

• Uncommitted changes are saved in the Rollback Tablespace.

• Redo Log files save all changes made in the Tablespaces.

• To start and/or recover from a system failure, the

Control Files

are read.

© 2002 by Prentice Hall

40

Archivelog

• If Oracle is running in redo log files.

information.

ARCHIVELOG mode, backup copies are made of the • Otherwise, the redo log files are periodically overwritten with new © 2002 by Prentice Hall

Types of Failures

• Application Failure – When a program bug is encountered or when a program does not correctly respond to current system conditions.

• Instance Failure – When Oracle is unable to do what it needs to do.

• Media Failure – When a disk becomes inaccessible to Oracle.

41 © 2002 by Prentice Hall

42

Recovery of an Application Failure

• Oracle rolls back uncommitted changes.

© 2002 by Prentice Hall

43

Recovery of an Instance Failure

• Oracle would be restarted using the following sequence… – Read the Control File – Restore system to last known valid state – Roll forward changes not in system (replay the Redo Log Files) © 2002 by Prentice Hall

44

Recovery from a Media Failure

• Restore system from Backup • Read the Control File • Roll forward changes using the Archive Log Files (from the ARCHIVELOG) • Roll forward changes from the on-line Log Files (the most recent versions of the Logs) © 2002 by Prentice Hall

Types of Recoveries

• Consistent Backup – After the restoration, delete all uncommitted activities – This ensures consistency, may lose recent changes • Inconsistent Backup – After the restoration, all uncommitted activities remain 45 © 2002 by Prentice Hall

Database Processing

Eighth Edition

Chapter 14 Networks, Multi-Tier Architectures, and XML

46 David M. Kroenke © 2002 by Prentice Hall

Networks

• A

network

is a collection of computers that communicate with one another using standard sets of rules, called

protocols

• Common Network Environments: – Internet – Intranet – Wireless Network Access 47 © 2002 by Prentice Hall

48

The Internet

• Internet network of networks spanning the globe - a publicly accessible • Uses communications protocol called

Transmission Control Program/Internet Protocol (TCP/IP)

© 2002 by Prentice Hall

49

Key Dates for the Internet

• The Internet was born in the 1960’s by the US armed services and was called ARPANET • HTTP: HyperText Transfer Protocol (used to create Web Pages) was created in 1989 by CERN – Key HTTP characteristics: • Request-based (waits for user action) • Stateless (does not sequence or remember activities) © 2002 by Prentice Hall

50

HTTP: Stateless Property

• In applications development, you may often wish to save the application state.

• Several Internet tools exist to help accomplish this: – Microsoft Internet Information Server (IIS) – Microsoft Active Server Pages (ASP) – Java Servlets with Java ServerPages (JSP) © 2002 by Prentice Hall

51

The Intranet

• Some organizations use Internet technologies to create their own privately accessible network called an

intranet

.

• If a connection to the Internet does exists, it does so through a than the Internet

firewall

• An intranet is almost always faster © 2002 by Prentice Hall

52

Firewall

• Firewall - a security gateway that protects an organization from unauthorized access via the Internet • Consists of software and sometimes hardware components © 2002 by Prentice Hall

53

Wireless Network Access

• Due to less reliability, inferior screen displays, and slower transfer rates, the traditional wired protocols are not appropriate for wireless environments.

• A few protocols have been developed which allow wireless devices to communicate via the Internet: – Wireless Application Protocol (WAP) – Wireless Markup Language (WML) © 2002 by Prentice Hall

54

Multi-tier Architectures

• Multi-tier Architectures – Tiers are the number of computers (serving a like function) that a user must use to satisfy his/her request. – Common tiers include Web server and database server.

© 2002 by Prentice Hall

55

A Three-Tier Architecture

© 2002 by Prentice Hall

56

Functions of Tiers

© 2002 by Prentice Hall

57

Processing at the Different Tiers

• Since each tier serves a different function, each tier may have a different operating system and different application software offerings.

© 2002 by Prentice Hall

58

Processing

• Client Processing – Using the browser (e.g., Netscape Navigator) • Server Processing – Using Server Software (e.g., ASP) © 2002 by Prentice Hall

59

Windows 2000 Web Server Languages • JavaScript • VBScript • Perl • ActiveX Control • Java

© 2002 by Prentice Hall

Standards & Languages Common With MS Web Server

60 © 2002 by Prentice Hall

61

Unix/Linux Web Server Environment

• JavaScript • Java Applets • Java Servlets • Java Server Pages • Perl • Java • CGI © 2002 by Prentice Hall

62

N-Tier Processing

• The 3-Tier architecture may be extended to include additional tiers.

• This produces a distributed processing model using various servers on the Internet © 2002 by Prentice Hall

Markup Languages

• Markup Languages specify the appearance and behavior of Web Pages • Markup Language flavors: – HTML – a subset of the SGML – DHTML – RDS/ADO – XML are used to 63 © 2002 by Prentice Hall

HTML

• HyperText Markup Language – PROS • Simple • Standardized – CONS • Static content • Limited connectivity • Mixed structure/content 64 © 2002 by Prentice Hall

DHTML

• Dynamic HyperText Markup Language • Encapsulates the entire HTML command set • Provides access to objects on the page using the Document Object Model (DOM) • Allows for Cascading Style Sheets (CSS) 65 © 2002 by Prentice Hall

Data Services

• Data services allow Web pages to exchange data with databases • RDS • ADO is a set of ObjectX controls – The data exchanges must be relatively simple is a set of ActiveX Data Controls – These data exchanges may be more complex 66 © 2002 by Prentice Hall

67

Extensible Markup Language –XML

• XML clearly separates content from structure and allows developers to easily define their own elements.

• Rather than hard-coding Web pages, you create rules that govern how the document should look. Then merge the structure and the content files. So, the very nature of XML is dynamic.

© 2002 by Prentice Hall

68

Document Type Declaration – DTD

• A

DTD

defines the data content and may provide the data values • While a DTD is desirable, it is not mandatory – XML documents using DTDs are termed – XML documents not using DTDs are termed

type-valid documents not-type-valid documents

© 2002 by Prentice Hall

69

XML & CSS

• Similar to DHTML,

Cascading Style Sheets (CSS)

may be used with XML documents to present a consistent, standardized Web site.

© 2002 by Prentice Hall

70

Extensible Style Language Transformation –XMLT

• XMLT is used to transform one document into another document © 2002 by Prentice Hall

XML Schema

• XML Schema is the next generation of DTD • The schema itself is an XML document • A W3 standard is currently being developed • A document that conforms to an XML Schema is termed schema-valid.

71 © 2002 by Prentice Hall

72

XML Schema Concepts

• Simple Elements – Consist of a single content value • Complex Elements – Consist of multiple content values © 2002 by Prentice Hall

73

XML Namespaces

• Namespaces define where to look for files • An XML document may have: – Up to one default namespace – Many labeled namespaces • Naming conventions: – Must be unique within all schemas – Typically resembles a URL, but is not a URL © 2002 by Prentice Hall

74

Wireless Application Protocol (WAP)

• WAP has been developed to facilitate Web development for wireless devices such as Personal Data Assistants (PDA) or cellular phones © 2002 by Prentice Hall

75

WAP Server

• A • A

WAP Server

transforms XML documents into Wireless Markup Language (WML) – WML is a subset of XML

WML Scripting Language

also exists © 2002 by Prentice Hall

76

XML and Database Applications

• Any document that can process a DTD or XML Schema document can correctly interpret any arbitrary database view • XML can easily process multiple multi-valued paths (several SQL statements would be required) © 2002 by Prentice Hall

77

XML and Database Applications

• The separation of structure and content allows for: – The same data to be displayed in many different ways – The same structure (report) may be regenerated many times with different/updated data.

– Permits document validation checking © 2002 by Prentice Hall

78

OASIS

– Document structures may be published and made publicly available – Organization for the Advancement of

Structured Information Standards (OASIS)

: • A clearinghouse for XML publications and schema standards © 2002 by Prentice Hall

79

DBMS Integration of XML

– Oracle • XML DOM parser • Xpath • XSQL – SQL Server • XML DOM parser • Xpath • ADO • ASP © 2002 by Prentice Hall

80

Web-based databases

© 2002 by Prentice Hall

81

Types of databases

• Textual databases • Semi-structured databases © 2002 by Prentice Hall

82

Indexing textual data

• Inverted files • Boolean queries • Signature files • Signature S 1 matches signature S 2 S 2 &S 1 =S 2 if © 2002 by Prentice Hall

83

XML-QL

© 2002 by Prentice Hall

XML-QL

Two slides from Johannes Gehrke, Cornell University “(x+y)^2” x y 2 WHERE $1 in “www.booklist.com/books.xml

CONSTRUCT $1 84 © 2002 by Prentice Hall

85

XML-QL (continued)

WHERE $b IN “www.booklist.com/books.xml”, $n $p in $e CONSTRUCT $p WHERE $l IN $n CONSTRUCT $l © 2002 by Prentice Hall

XML-QL (continued)

86 © 2002 by Prentice Hall

87

XML-QL (continued)

WHERE Addison Wesley $t $a IN "www.a.b.c/bib.xml" CONSTRUCT $a © 2002 by Prentice Hall

88

XML-QL (continued)

WHERE Addison Wesley $t</> <author> $a</> </> IN "www.a.b.c/bib.xml" CONSTRUCT $a © 2002 by Prentice Hall</p> <a id="p89" href="#"></a> <p>89</p> <h3>XML-QL (continued)</h3> <p>WHERE <book> <publisher><name>Addison-Wesley</></> <title> $t</> <author> $a</> </> IN "www.a.b.c/bib.xml" CONSTRUCT <result> <author> $a</> <title> $t</> </> © 2002 by Prentice Hall</p> <a id="p90" href="#"></a> <h3>XML-QL (continued)</h3> <p>90 <bib> <book year="1995"> <!-- A good introductory text --> <title> An Introduction to Database Systems Date Addison-Wesley Foundation for Object/Relational Databases: The Third Manifesto Date Darwen Addison-Wesley © 2002 by Prentice Hall

XML-QL (continued)

Date An Introduction to Database Systems Date Foundation for Object/Relational Databases: The Third Manifesto 91 Darwen Foundation for Object/Relational Databases: The Third Manifesto © 2002 by Prentice Hall

92

XML-QL (continued)

WHERE $p IN "www.a.b.c/bib.xml", $t</>, <publisher><name>Addison-Wesley</>> IN $p CONSTRUCT <result> <title> $t </> WHERE <author> $a </> IN $p CONSTRUCT <author> $a</> </> © 2002 by Prentice Hall</p> <a id="p93" href="#"></a> <h3>XML-QL (continued)</h3> <p><result> <title> An Introduction to Database Systems Date Foundation for Object/Relational Databases: The Third Manifesto Date Darwen 93 © 2002 by Prentice Hall

94

XML-QL (continued)

WHERE

$f // firstname $f $l // lastname $l CONTENT_AS $a IN "www.a.b.c/bib.xml" $f // join on same firstname $f $l // join on same lastname $l IN "www.a.b.c/bib.xml", y > 1995 CONSTRUCT
$a © 2002 by Prentice Hall

95

XML-QL (continued)

© 2002 by Prentice Hall

96

XML-QL (continued)

© 2002 by Prentice Hall

97

XML-QL (continued)

John Smith . . .

... 1995
© 2002 by Prentice Hall

98

XML-QL (continued)

© 2002 by Prentice Hall

99

XML-QL (continued)

WHERE

$n IN "abc.xml” WHERE
</> ELEMENT_AS $t </>, <person ID=$i> <lastname> </> ELEMENT_AS $l </> CONSTRUCT <result> $t $l</> © 2002 by Prentice Hall</p> <a id="p100" href="#"></a> <h3>Scalar values</h3> <p><title>A Trip to <titlepart> the Moon </titlepart> NOT!

<CDATA> A Trip to </CDATA><titlepart><CDATA> the Moon</CDATA></titlepart> YES 100 © 2002 by Prentice Hall

101

Tag variables

WHERE <$p> $t 1995 <$e> Smith IN "www.a.b.c/bib.xml", $e IN {author, editor} CONSTRUCT <$p> $t <$e> Smith © 2002 by Prentice Hall

102

Transforming data

© 2002 by Prentice Hall

103

Transforming data (cont’d)

WHERE <$> $fn $ln $t </> </> IN "www.a.b.c/bib.xml", CONSTRUCT <person ID=PersonID($fn, $ln)> <firstname> $fn </> <lastname> $ln </> <publicationtitle> $t </> </> © 2002 by Prentice Hall</p> <a id="p104" href="#"></a> <p>104</p> <h3>Integrating data from different sources</h3> <p>WHERE <person> <name></> ELEMENT_AS $n <ssn> $ssn</> </> IN "www.a.b.c/data.xml", <taxpayer> <ssn> $ssn</> <income></> ELEMENT_AS $i </> IN "www.irs.gov/taxpayers.xml" CONSTRUCT <result> $n $i </> © 2002 by Prentice Hall</p> <a id="p105" href="#"></a> <h3>Query blocks</h3> <p>105 WHERE <$e> <title> $t </> </> <year> 1995 </> </> CONTENT_A $p IN "www.a.b.c/bib.xml" CONSTRUCT <result ID=ResultID($p)> <title> $t </> </> { WHERE $e = "journal-paper", <month> $m </> IN $p CONSTRUCT <result ID=ResultID($p)> <month> $m </> } { WHERE $e = "book", <publisher>$q </> IN $p CONSTRUCT <result ID=ResultID($p)> <publisher>$q </> </> } © 2002 by Prentice Hall</p> <a id="p106" href="#"></a> <p>106</p> <h3>WSQ</h3> <p>© 2002 by Prentice Hall</p> <a id="p107" href="#"></a> <h3>Web-supported queries</h3> <p>SIGMOD2000 (Goldman and Widom) WebPages (SearchExp,T1,T2,…,Tn,URL,Rank, Date) SELECT NAME, COUNT FROM STATES, WEBCOUNT WHERE NAME = T1 ORDER BY COUNT DESC 107 © 2002 by Prentice Hall</p> <a id="p108" href="#"></a> <p>108</p> <h3>KDD: Data Mining</h3> <p>© 2002 by Prentice Hall</p> <a id="p109" href="#"></a> <h3>The big problem</h3> <p>• Billions of records • A small number of interesting patterns • “Data rich but information poor” 109 © 2002 by Prentice Hall</p> <a id="p110" href="#"></a> <h3>Data mining</h3> <p>• Knowledge discovery • Knowledge extraction • Data/pattern analysis 110 © 2002 by Prentice Hall</p> <a id="p111" href="#"></a> <p>111</p> <h3>Types of source data</h3> <p>• Relational databases • Transactional databases • Web logs • Textual databases © 2002 by Prentice Hall</p> <a id="p112" href="#"></a> <h3>Association rules</h3> <p>• 65% of all customers who buy beer and tomato sauce also buy pasta and chicken wings • Association rules: X  Y 112 © 2002 by Prentice Hall</p> <a id="p113" href="#"></a> <h3>Association analysis</h3> <p>• IF 20 < age < 30 AND 20K < INCOME < 30K • THEN – Buys (“CD player”) • SUPPORT = 2%, CONFIDENCE = 60% 113 © 2002 by Prentice Hall</p> <a id="p114" href="#"></a> <h3>Basic concepts</h3> <p>• Minimum support threshold • Minimum confidence threshold • Itemsets • Occurrence frequency of an itemset 114 © 2002 by Prentice Hall</p> <a id="p115" href="#"></a> <h3>Association rule mining</h3> <p>• Find all frequent itemsets • Generate strong association rules from the frequent itemsets 115 © 2002 by Prentice Hall</p> <a id="p116" href="#"></a> <h3>Support and confidence</h3> <p>• Support (X) • Confidence (X  Support (X) Y) = Support(X+Y) / 116 © 2002 by Prentice Hall</p> <a id="p117" href="#"></a> <p>117</p> <p><b>TID</b></p> <p>T100 T200 T300 T400 T500 T600 T700 T800 T900</p> <h3>Example</h3> <p><b>List of items IDs</b></p> <p>I1, I2, I5 I2, I4 I2, I3 I1, I2, I4 I1, I3 I2, I3 I1, I3 I1, I2, I3, I5 I1, I2, I3 © 2002 by Prentice Hall</p> <a id="p118" href="#"></a> <h3>Example (cont’d)</h3> <p>• Frequent itemset <i>l </i>= {I1, I2, I5} • I1 AND I2  I5 C = 2/4 = 50% • I1 AND I5  • I2 AND I5  • I1  I2 I1 I2 AND I5 • I2  • I3  I1 AND I5 I1 AND I2 118 © 2002 by Prentice Hall</p> <a id="p119" href="#"></a> <p>119</p> <p><b>TID</b></p> <p>T100 T200 T300 T400</p> <h3>Example 2</h3> <p><b>date items</b></p> <p>10/15/99 {K, A, D, B} 10/15/99 {D, A, C, E, B} 10/19/99 {C, A, B, E} 10/22/99 {B, A, D} min_sup = 60%, min_conf = 80% © 2002 by Prentice Hall</p> <a id="p120" href="#"></a> <h3>Correlations</h3> <p>• Corr (A,B) = P (A OR B) / P(A) P (B) • If Corr < 1: A discourages B (negative correlation) • (lift of the association rule A  B) 120 © 2002 by Prentice Hall</p> <a id="p121" href="#"></a> <p>121</p> <h3>Contingency table</h3> <p>Video Game 4,000 ^Video 2,000 Sum 6,000 ^Game Sum 3,500 7,500 500 4,000 2,500 10,000 © 2002 by Prentice Hall</p> <a id="p122" href="#"></a> <h3>Example</h3> <p>• P({game}) = 0.60</p> <p>• P({video}) = 0.75</p> <p>• P({game,video}) = 0.40</p> <p>• P({game,video})/(P({game})x(P({video}) ) = 0.40/(0.60 x 0.75) = 0.89</p> <p>122 © 2002 by Prentice Hall</p> <a id="p123" href="#"></a> <p>123</p> <h3>Example 2</h3> <p>hotdog s hamburgers 2000 ^hotdo gs 500 Sum 2500 ^hamburger s Sum 1000 3000 1500 2000 2500 5000 © 2002 by Prentice Hall</p> <a id="p124" href="#"></a> <p>124</p> <h3>Classification using decision trees</h3> <p>• Expected information need • I (s 1 , s 2 , …, s m ) = S • s = data samples • m = number of classes p i log (p i ) © 2002 by Prentice Hall</p> <a id="p125" href="#"></a> <p>125 1 2 3 4 5 6 7 8 9 10 11 12 13 14</p> <p><b>RID Age</b></p> <p><= 30 <= 30 31 .. 40 > 40 > 40 > 40 31 .. 40 <= 30 <= 30 > 40 <= 30 31 .. 40 31 .. 40 > 40</p> <p><b>Income</b></p> <p>High High High Medium Low Low Low Medium Low Medium Medium Medium High Medium</p> <p><b>student</b></p> <p>No No No No Yes Yes Yes No Yes Yes Yes No Yes no</p> <p><b>credit buys?</b></p> <p>Fair No Excellent No Fair Yes Fair Fair Excellent No Excellent Yes Fair Yes Yes No Fair Fair Excellent Yes Excellent Yes Fair Yes Yes Yes excellent no © 2002 by Prentice Hall</p> <a id="p126" href="#"></a> <h3>Decision tree induction</h3> <p>• I(s 1 ,s 2 ) = I(9,5) = = - 9/14 log 9/14 – 5/14 log 5/14 = = 0.940</p> <p>126 © 2002 by Prentice Hall</p> <a id="p127" href="#"></a> <p>127</p> <h3>Entropy and information gain</h3> <p>S s 1j ,…,s mj ) Entropy = expected information based on the partitioning into subsets by A Gain (A) = I (s 1 ,s 2 ,…,s m ) – E(A) © 2002 by Prentice Hall</p> <a id="p128" href="#"></a> <h3>Entropy</h3> <p>• Age <= 30 s 11 = 2, s 21 = 3, I(s 11 , s 21 ) = 0.971</p> <p>• Age in 31 .. 40 s 12 = 4, s 22 = 0, I (s 12 ,s 22 ) = 0 • Age > 40 s 13 = 3, s 23 = 2, I (s 13 ,s 23 ) = 0.971</p> <p>128 © 2002 by Prentice Hall</p> <a id="p129" href="#"></a> <h3>Entropy (cont’d)</h3> <p>• E (age) = 5/14 I (s11,s21) + 4/14 I (s12,s22) + 5/14 I (S13,s23) = 0.694</p> <p>• Gain (age) = I (s1,s2) – E(age) = 0.246</p> <p>• Gain (income) = 0.029, Gain (student) = 0.151, Gain (credit) = 0.048</p> <p>129 © 2002 by Prentice Hall</p> <a id="p130" href="#"></a> <p>130</p> <h3>Final decision tree</h3> <p>age > 40 no no student yes yes 31 .. 40 yes excellent no credit fair yes © 2002 by Prentice Hall</p> <a id="p131" href="#"></a> <h3>Other techniques</h3> <p>• Bayesian classifiers • X: age <=30, income = medium, student = yes, credit = fair • P(yes) = 9/14 = 0.643</p> <p>• P(no) = 5/14 = 0.357</p> <p>131 © 2002 by Prentice Hall</p> <a id="p132" href="#"></a> <h3>Example</h3> <p>• P (age < 30 | yes) = 2/9 = 0.222</p> <p>P (age < 30 | no) = 3/5 = 0.600</p> <p>P (income = medium | yes) = 4/9 = 0.444</p> <p>P (income = medium | no) = 2/5 = 0.400</p> <p>P (student = yes | yes) = 6/9 = 0.667</p> <p>P (student = yes | no) = 1/5 = 0.200</p> <p>P (credit = fair | yes) = 6/9 = 0.667</p> <p>P (credit = fair | no) = 2/5 = 0.400</p> <p>132 © 2002 by Prentice Hall</p> <a id="p133" href="#"></a> <h3>Example (cont’d)</h3> <p>• P (X | yes) = 0.222 x 0.444 x 0.667 x 0.667 = 0.044</p> <p>• P (X | no) = 0.600 x 0.400 x 0.200 x 0.400 = 0.019</p> <p>• P (X | yes) P (yes) = 0.044 x 0.643 = 0.028</p> <p>• P (X | no) P (no) = 0.019 x 0.357 = 0.007</p> <p>• Answer: yes/no?</p> <p>133 © 2002 by Prentice Hall</p> <a id="p134" href="#"></a> <h3>More types of data mining</h3> <p>• Classification and prediction • Cluster analysis • Outlier analysis • Evolution analysis 134 © 2002 by Prentice Hall</p> </div> </section> </div> </div> </div> </main> <footer> <div class="container mt-3"> <div class="row justify-content-between"> <div class="col"> <a href="/"> <img src="/theme/studyslide/static/logo-slideum.png" /> </a> </div> </div> <div class="row mt-3"> <ul class="col-sm-6 list-unstyled"> <li> <h6 class="mb-3">Company</h6> <li> <i class="fa fa-location-arrow"></i> Nicosia Constantinou Palaiologou 16, Palouriotissa, 1040 <li> <i class="fa fa-phone"></i> +357 64-733-402 <li> <i class="fa fa-envelope"></i> info@slideum.com </ul> <ul class="col-6 col-sm-3 list-unstyled"> <li> <h6 class="mb-3">Links</h6> <li> <a href="/about">About</a> <li> <a href="/contacts">Contact</a> <li> <a href="/faq">Help / FAQ</a> </ul> <ul class="col-6 col-sm-3 list-unstyled"> <li> <h6 class="mb-3">Legal</h6> <li> <a href="/terms">Terms of Service</a> <li> <a href="/privacy">Privacy policy</a> <li> <a href="/page.html?code=public.usefull.cookie">Cookie policy</a> <li> <a href="/page.html?code=public.usefull.disclaimer">Disclaimer</a> </ul> </div> <hr> <p>slideum.com © 2026, Inc. All rights reserved.</p> </div> </footer> <div class="modal directory" id="directory-modal"> <div class="modal-dialog"> <div class="modal-content"> <div class="modal-header"> <h5 class="modal-title">Directory</h5> <button class="close" type="button" data-dismiss="modal">×</button> </div> <div class="modal-body"></div> </div> </div> </div> <script src="/theme/common/static/jquery@3.6.0/dist/jquery.min.js"></script> <script src="/theme/common/static/jquery_extra/dist/jquery-extra.js"></script> <script src="/theme/common/static/popper.js@1.16.1/dist/umd/popper.min.js"></script> <script src="/theme/common/static/bootstrap@4.6.0/dist/js/bootstrap.min.js"></script> <script> var __path_directory = [ ] !function __draw_directory(data, root, uuid) { var ul = $('<ul>', uuid && { id: 'category' + uuid, class: !__path_directory.includes(uuid) ? 'collapse' : null }); for (var item in data) { var li = $('<li>').appendTo(ul); if (item = data[item], item.children) { li.append('<a href=#category' + item.id + ' data-toggle=collapse>') __draw_directory(item.children, li, item.id); } else { li.append('<a href=' + item.url + '>'); } var a = $('> a', li).addClass('item').text(item.name) .append($('<a class="link fa fa-external-link" href=' + item.url + '>')); if (item.id === +__path_directory.slice(-1)) { a.addClass('active'); } /* if (item.id !== __path_directory[0]) { a.addClass('collapsed'); } */ } root.append(ul); } ([{"id":1,"name":"Food and cooking","url":"/catalog/Food+and+cooking","children":null},{"id":2,"name":"Education","url":"/catalog/Education","children":null},{"id":3,"name":"Healthcare","url":"/catalog/Healthcare","children":null},{"id":4,"name":"Real estate","url":"/catalog/Real+estate","children":null},{"id":5,"name":"Religion ","url":"/catalog/Religion+","children":null},{"id":6,"name":"Science and nature","url":"/catalog/Science+and+nature","children":null},{"id":7,"name":"Internet","url":"/catalog/Internet","children":null},{"id":8,"name":"Sport","url":"/catalog/Sport","children":null},{"id":9,"name":"Technical documentation","url":"/catalog/Technical+documentation","children":null},{"id":10,"name":"Travel","url":"/catalog/Travel","children":null},{"id":11,"name":"Art and Design","url":"/catalog/Art+and+Design","children":null},{"id":12,"name":"Automotive","url":"/catalog/Automotive","children":null},{"id":13,"name":"Business","url":"/catalog/Business","children":null},{"id":14,"name":"Government","url":"/catalog/Government","children":null}], $('#directory-aside')); var __root_directory = $('#directory-aside > ul'); $('#directory-aside') .on('show.bs.collapse', function() { //console.log('show.collapse') }) .on('hide.bs.collapse', function() { //console.log('hide.collapse') }); $('#directory-modal') .on('show.bs.modal', function() { $('[class$="body"]', this).prepend(__root_directory); }) .on('hide.bs.modal', function() { $('#directory-aside').prepend(__root_directory); }); $('.directory-mobile').on('click', function(e) { e.preventDefault(); }); $('.directory .link').on('click', function(e) { e.stopPropagation(); }); </script> <script> function scrollToViewport() { $('html, body').stop().animate( { scrollTop: $('.navbar').outerHeight() }, 1000); } setTimeout(scrollToViewport, 1000); $(window).on('orientationchange', scrollToViewport); $('[data-toggle="tooltip"]').tooltip(); </script> <script async src="//s7.addthis.com/js/300/addthis_widget.js#pubid=#sp('addthis_pub_id')"></script> <script type="text/javascript"> (function (d, w, c) { (w[c] = w[c] || []).push(function() { try { w.yaCounter28397281 = new Ya.Metrika({ id:28397281 }); } catch(e) { } }); var n = d.getElementsByTagName("script")[0], s = d.createElement("script"), f = function () { n.parentNode.insertBefore(s, n); }; s.type = "text/javascript"; s.async = true; s.src = (d.location.protocol == "https:" ? "https:" : "http:") + "//mc.yandex.ru/metrika/watch.js"; if (w.opera == "[object Opera]") { d.addEventListener("DOMContentLoaded", f, false); } else { f(); } })(document, window, "yandex_metrika_callbacks"); </script> <noscript><div><img src="//mc.yandex.ru/watch/28397281" style="position:absolute; left:-9999px;" alt="" /></div></noscript> <link rel="stylesheet" type="text/css" href="//cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.1.0/cookieconsent.min.css" /> <style> @media screen and (max-width: 768px) { .cc-revoke { display: none; }} </style> <script src="//cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.1.0/cookieconsent.min.js"></script> <script> window.addEventListener("load", function() { window.cookieconsent.initialise( { content: { href: "https://slideum.com/dmca" }, location: true, palette: { button: { background: "#fff", text: "#237afc" }, popup: { background: "#007bff" }, }, position: "bottom-right", revokable: true, theme: "classic", type: "opt-in" })}); </script> </body> </html>