Transcript Document

YAML: A New Language for Data
INGY presents:
YAML@
YAPC
YAML: A New Language for Data
&whois("Brian Ingerson");
•
•
•
•
•
•
•
•
•
Libraries and Mainframes
Assembler to Perl
Term::Scraper
Inline
YAML
use YourBrainForOnceDude;
ActiveState
FatPort
DataBuilt
YAML: A New Language for Data
What is YAML?
•
Your Average Meat Loaf?
• Yasser Arafat Making Lasers?
• You'll Argue My Logic?
• YAGNI Ass Mutha Load?
• whY Are Mongers Laughing?
• Yet Another Module Legend?
• Yet Another Markup Language
• ???
• YAML Ain’t Markup Language!
YAML: A New Language for Data
What is YAML?
•
•
•
•
YAML is a Data Serialization Language.
A New Language for Data
Modeled after Perl-like structures
But not just intended for Perl
• XML is a Markup Language
• Sort of...
• We use XML mostly for serialization
YAML: A New Language for Data
YAML Design Goals
YAML design goals:
- YAML documents are very readable by humans.
- YAML interacts well with scripting languages.
- YAML uses host languages native data structures.
- YAML has a consistent information model.
- YAML enables stream-based processing.
- YAML is expressive and extensible.
- YAML is easy to implement.
(cat | ysh)
YAML: A New Language for Data
Ingy's Design Goals
•
•
•
•
•
To achieve Human/Computer Nirvana
High Data/Markup (Signal/Noise) Ratio
To be the world's best Data::Dumper
To be great for Configs, Caches, and Logs
One Serialization to Rule Them All...
... and in the YAML bind them.
Other Goals:
• Unicode based
• Speedy parsing
• Alternative to XML (for some applications)
YAML: A New Language for Data
Where YAML Came From
•
•
•
•
•
•
Inline.pm
Data::Denter
Clark Evans
Oren Ben-Kiki
sml-dev
yaml-core mailing list
•
•
•
•
mini P5P
2000+ messages last year
irc.debian.org #yaml
Summits
YAML: A New Language for Data
Data::Denter and Inline
md5 : cc9b569052f4daa5b343b1dcb94dd2bc
name : e_cc9b
language : C
date_compiled : Wed Jun 12 12:48:00 2002
inline_version : 0.43
ILSM : %
module : Inline::C
suffix : bundle
type : compiled
Config : %
apiversion : 5.005
archname : darwin
ccflags : -g -pipe -pipe -fno-common
osname : darwin
osvers : 1.4
so : dylib
version : 5.6.0
YAML: A New Language for Data
Influences
--scripting languages:
- Perl
- Python
- C
- Java
standards:
- RFC0822 (MAIL)
- RFC1866 (HTML)
- RFC2045 (MIME)
- RFC2396 (URI)
protocols:
- SAX
- SOAP
- XML
YAML: A New Language for Data
YAML Syntax Basics
•
•
•
•
•
Mappings
Sequences
Streams and Documents
Comments
Scalars
•
•
•
•
Simple
Quoted
Block
Folded
• Wiki
• Escaping
• Anchors & Aliases
YAML: A New Language for Data
Mappings
• A YAML mapping is like a Perl hash
• Unordered Key/Value pairs
• Separated by ': ' (space is mandatory)
--name: Benjamin
rank: Private
serial number: 1234567890
12:34 PM: My favorite time
YAML: A New Language for Data
Sequences
• A YAML sequence is like a Perl array
• An ordered collection of data
• YAML has a bullet like syntax '- '
--- red
- white
- blue
- pinko
YAML: A New Language for Data
A YAML Grocery List
--Fruits:
- Apples
- Tomatoes
Veggies:
- Spinach
- Broccoli
Meats:
- Burgers
- Shrimp
Household:
- Candles
- Incense
- Toilet Duck
YAML: A New Language for Data
The Matrix
---
3
5
7
0
0
7
9
1
1
YAML: A New Language for Data
Outline
- Intro
Part 1:
- Up
- Down
- Side to Side
- Part 2:
- Here
- There
- Underwear
- Part 3:
- The Good
- The Bad
- The Ingy
YAML: A New Language for Data
Comments
• Comments/blank lines can go almost anywhere
• Must not be ambiguous with content
• Comments begin '# ' (almost like Perl)
# comment before document
--- #DIRECTIVE # comment
foo: bar # inline comment
phone: number #555-1234
### Comment
fact: fiction
--blue: bird
# Comment
YAML: A New Language for Data
Scalars
• A YAML scalar is like a Perl scalar
• YAML has many scalar forms
• TMTOWTDI
--simple: look ma, no quotes
quoted:
- 'Single quoted. Like Perl, no escapes'
- "Double quotes.\nLike Perl, has escapes"
- |
A YAML block scalar.
Much like Perl's
here-document.
YAML: A New Language for Data
Simple (unquoted) Scalar
• Can use for almost all simple values
• Some content limitations
--simple key: simple value
this value: can span multiple lines
but the key cannot. it would need quotes
stuff:
- foo
- 42
- 3.14
- 192.168.2.98
- m/^(.*)\//
YAML: A New Language for Data
Single Quoted Scalar
• No escapes (like Perl)
• Use when simple won't work
--'key contains: colon': '$19.99'
or: ' value has leading/trailing whitespace '
'key spans
lines': 'double ticks \ for ''escaping'''
YAML: A New Language for Data
Double Quoted Scalar
• Allows escaping (like Perl)
--The spec says: "The double quoted style
variant adds escaping to the 'single quoted'
style variant."
like this: "null->\z newline->\n bell->\a
smiley->\u263a"
self escape: "Brian \"Ingy\" Ingerson"
YAML: A New Language for Data
Block Scalar
• Blocks are like here-docs on vitamins
• Newlines are hard
• Termination is determined by indentation
--what is this: |
is it: a YAML mapping
or just: a string
chomp me: |sub foo {
print "Love me do!";
}
YAML: A New Language for Data
Embedding YAML in YAML
• No Escaping of Markup Required
• Simply use the Block Form
• YAML Test Suite does this
--- #YAML:1.0
old doc: |
--- #YAML:1.0
tools:
- XML
- XSLT
new doc: |
--- #YAML:1.0
tools:
- YAML
- cYATL
YAML: A New Language for Data
Folded Scalar
• Folded scalars work like HTML
• Single newlines indicate a space character.
• Empty lines mean a hard newline
--- >
Copyright © 2001 Brian Ingerson, Clark
Evans & Oren Ben-Kiki, all rights
reserved. This document may be freely
copied provided that it is not modified.
Next paragraph.
- foo
YAML: A New Language for Data
Wiki Content
• Similar to POD or some Wikis
• Indented text preserves line endings
--The YAML Specification starts out by saying: >
YAML(tm) (rhymes with "camel") is a straightforward
machine parsable data serialization format designed
for human readability and interaction with
scripting languages such as Perl and Python.
YAML documents are very readable by humans.
YAML interacts well with scripting languages.
YAML uses host languages' native data
structures.
Please join us, the mailing list is at SourceForge.
YAML: A New Language for Data
Nested Scalar Key
• Sometimes you want fancy blocks for map keys
• Know what I mean?
--? >+
Even a key can:
1) Be Folded
2) Have Wiki
: cool, eh?
YAML: A New Language for Data
Anchors and Aliases
• YAML can handle duplicate and recursive data
• It uses an alias mechanism
• Aliases do not span documents
--Hey
-
Jude: &chorus
na, na, na,
&4 na, na, na, na,
*4
Hey Jude.
*chorus
YAML: A New Language for Data
A Hard Example
• Here is an example of preserving hard refs
• Not sexy, but neither are hard refs...
use YAML;
$a=\\\\\\\\"foo";
$b=$$$$a;
print Dump [$a,$b],[$b,$a];
YAML: A New Language for Data
Using YAML.pm
• Just like Data::Dumper
• perl -MData::Dumper -e 'print Dumper \%INC'
• perl -MYAML -e 'print Dump \%INC'
• 10 character savings over D::D
• Basic API
• Dump
• Load
• Other Functions:
• DumpFile
• LoadFile
• freeze(), thaw() # POE
• Object Oriented # Not discussed
YAML: A New Language for Data
YAML vs Data::Dumper
• YAML Pros:
•
•
•
•
•
•
Cleaner format
Safe deserialization
Hashes sorted by default
Escapes unprintables
Supports GLOB, CODE, Regexp and Filehandle
Interacts with Ruby, Python, Java etc
• Data::Dumper Pros:
• Familiar Perl Syntax
• Fast C implementation
• Mature
YAML: A New Language for Data
Using YAML in Perl Code
my $calendar = Load <<'...';
jan: [january,
31]
feb: [february, 28]
mar: [march,
31]
apr: [april,
30]
may: [may,
31]
jun: [june,
30]
jul: [july,
31]
aug: [august,
31]
sep: [september, 30]
oct: [october,
31]
nov: [november, 30]
dec: [december, 31]
...
$july_days = $calendar->{jul}[1];
YAML: A New Language for Data
Some Real YAML Applications
• yaml2outline
• phone2yaml
• xyx
• YAML.pm uses YAML!
• Changes
• Test suite
• perl test-prd.pl
YAML: A New Language for Data
A Quick Tour of the YAML Spec
•
http://www.yaml.org/spec
1. Intro
2. Preview
•
Nice Examples
3. Information Models
4. Serialization Syntax
•
160 productions
5. Transfer Methods
YAML: A New Language for Data
Intermediate YAML Syntax
• Whitespace ...
• ... is the devil
•
•
•
•
•
•
•
Headers and Directives
Inline Collections
Complex Keys
Transfer Families
Implicit Typing
YAML Builtin Types
Perl Builtin Types
YAML: A New Language for Data
Whitespace
• There is a special room in the hot place reserved
for the person who invented tabs.
•
•
•
•
•
YAML has it harder than Python
By default YAML disallows TABS for indenting
But we have #TAB just in case
Modern editors will have YAML support
Keep computing tidy. Burn all tabs!
YAML: A New Language for Data
Indentation Rules
• Use same number of spaces for a given scope
--two:
- few
four:
- more
blocks:
- |
# Beware the TAB
sub spoon {
-------- fork
}
- |2
QTY DESC
PRICE
1) 3
Blind Mice
Free
2) 5
Dollar Bill
$10.00
YAML: A New Language for Data
Headers and Directives
•
•
•
•
Headers start all documents
Directives and top level qualifiers go there
Two directives: #YAML & #TAB
First document can be headerless
headerless: first document
--- #YAML:1.0 #TAB:NONE
--- >
folded top level scalar
--- &1
recurse: *1
--- simple header
YAML: A New Language for Data
Inline Collections
• So far YAML structure resembles Python
• Time for a little TMTOWTDI!!!
--seq: [ 14, 34, 55 ]
map: {purple: rain, blue: skies}
mixed: {sizes: [9, 11], shapes: [round]}
span: {players: [who, what, I don't know],
positions: [first, second, third]}
YAML: A New Language for Data
The Matrix Revisited
# Inline
--- [3, 5,
- [0, 0,
- [9, 1,
sequences make data more compact
7]
7]
1]
# Above is identical to below
--- [[3, 5, 7], [0, 0, 7], [9, 1, 1]]
# A 3D
--- [[3,
- [[0,
- [[9,
Matrix
5, 7], [0, 0, 7], [9, 1, 1]]
0, 7], [9, 1, 1], [3, 5, 7]]
1, 1], [3, 5, 7], [0, 0, 7]]
YAML: A New Language for Data
Complex Keys
• In Perl, all hash keys are strings
• YAML doesn't have that restriction
• YAML is Perl6 enabled!
--?
- Kane
- Kudra
: engaged
[Damian, Dominus]: engaging
YAML: A New Language for Data
Transfer Families
• In YAML, every node has a type
• These types can be user defined
• Every type has a unique URI
same:
perl:
-
42
!int 42
!yaml.org/int 42
!http://yaml.org/int 42
!perl/Foo::Bar {}
!perl.yaml.org/Foo::Bar {}
!http://perl.yaml.org/Foo::Bar {}
YAML: A New Language for Data
More Transfer Families
• You can define your own types
• '^' is used for abbreviating
• URI's may someday be real schema locations
domain specific document:
- !http://foobar.org/foo []
- !foobar.org/^foo
- !^bar xxx
- !^baz yyy
YAML: A New Language for Data
Implicit Typing
• Simple (unquoted) Strings have implicit types
• These types have predefined regexps
--- 42
- -3.14
- 6.02e+23
- 0xCAFEBABE
- 2001-09-11
- '2001-09-11'
- +
- (false)
- ~
- 123 Main St
#
#
#
#
#
#
#
#
#
#
integer
floating point
scientific notation
hexadecimal int
ISO8601 time
string
boolean true
alternate boolean
null (undef in Perl)
string
YAML: A New Language for Data
Roundtripping
•
•
•
•
•
•
•
•
YAML processes should preserve types
Especially within a language/application domain
Should warn otherwise
Python preserves Perl typeglobs (!perl/glob:)
Perl preserves !python/tuple
Gets complicated with complex keys, etc
YAML::Node is the Perl roundtripping object
A tied hash/array/scalar w/ YAML magic
YAML: A New Language for Data
YAML Builtin Types
• YAML has a good number of builtin types
• Some are implicit, some aren't
--- !str YAML, YAML, YAML!
- !int 42
- !float 0.707
- !time 2001-12-14T21:59:43.10-05:00
- !bool 1
- !null 0
- !binary MWYNG84BwwEeECcgggoBADs=
YAML: A New Language for Data
Perl Builtin Types
• Perl also has a number of official types
• These are under YAML/Ingy's control
• !http://perl.yaml.org/type
--- !perl/Foo::Bar {}
- !perl/@Foo::Bar []
- !perl/$Foo::Bar ''
- !perl/glob:
- !perl/code:
- !perl/ref:
- !perl/regexp:
- !perl/regexp:Foo::Bar
#
#
#
#
#
#
#
#
hash-based class
array-based class
scalar-based class
typeglob
code reference
hard reference
regular expression
blessed regexp
YAML: A New Language for Data
Introducing the YAML Shell
•
•
•
•
•
•
•
ysh - the YAML Shell
Perl => YAML
YAML => Perl
Term::Readline support
Logging
Great way to learn Perl
Distributed with YAML.pm
YAML: A New Language for Data
Fun With GLOBs in ysh
•
•
•
•
•
•
•
•
*foo
*INC
*ARGV
*$
*_
*STDIN
*YAML::
[*x, *y]
YAML: A New Language for Data
Perl's Symbol Table (The Full Monty!)
• ysh -uc -i4 -L
• *main::
• 10000+ lines
• recursive (main::)
• ISA, IO:, CODE: &6, ENV:,
Config:(tied!), INC:, SIG:,
Bits:, Offsets:, *|
YAML: A New Language for Data
More ysh Fun
•
•
•
•
•
•
•
•
•
open $fh, $0;
open $fh[0], $0;
seek $fh, 100, 0;
close $fh;
scalar `perl -h`
fork
< Changes
Roundtripping
$x = \$x;
YAML: A New Language for Data
The World Wide YAML Shell
• http://wwysh.yaml.org
• Regression Test Farm
• Multiple Implementations
YAML: A New Language for Data
Self Serializing Objects
• YAML::Node
• Family URI
• Ordered keys
• Foo::yaml_dump()
• Foo::yaml_load()
YAML: A New Language for Data
Advanced YAML Concepts
•
•
•
•
•
•
YAML::Node
Information Models
YAML Schemas
YPATH & YATL
YAML::Node
YAML::Bless()
YAML: A New Language for Data
YAML::Node
• Need more control?
• YAML::Node is the ticket
• Tied wrapper abound a plain hash, array or scalar
• Contains extra info:
• YAML Family
• Key order
• YAML::Bless($node)
• $y->bless($foo);
• Associates a ynode with a regular node
• YAML::Blessed($node)
• May be a way to support complex keys in Perl
YAML: A New Language for Data
Information Models
• Syntax Model
• parser/emitter
• Serial Model
• loader/dumper
• Generic Model
• application loader/dumpers
• Native Model
• user application (your script)
YAML: A New Language for Data
YAML and Schemas
• Key Order
• Optional Fields
• Validation (enums)
• Structure
YAML: A New Language for Data
YAML Schema
--- #YAML:1.0
NAME: AddressEntry
HASH:
- NAME: Name
HASH:
- NAME: First
- NAME: Last
OPTIONAL: yes
- NAME: EmailAddresses
ARRAY: yes
- NAME: Phone
ARRAY: yes
HASH:
- NAME: Type
OPTIONAL: yes
- NAME: Number
YAML: A New Language for Data
YAML Instance w/ Schema
--- #YAML:1.0
AddressEntry:
Name:
First: Brian
EmailAddresses:
- [email protected]
- [email protected]
Phone:
- Type: Work
Number: 604-333-4567
- Number: 843-444-5678
YAML: A New Language for Data
libyaml - The core YAML engine
•
•
•
•
•
One engine for Perl, Python, Tcl, Ruby, PHP
Maybe Java??
YAML.pm will bind using Inline!
Thank you Neil!
Thank you Clark!
YAML: A New Language for Data
YAML People
•
•
•
•
•
•
•
•
•
•
•
•
Clark Evans - Father of YAML - libyaml,Python
Oren Ben-Kiki - The Master Mind - Java
Brian Ingerson - Perl Watchdog - YAML.pm
Neil Watkiss - Boy Wonder - libyaml
Steve and Ryan - The XP factor - Python, Ruby
Rolf Veen - The German Connection - Java
Jason Diamond - The Lost Soul - C#
Neil Kandalgoankar - brev - YAML.py.old
Shane Caraveo - Mr 10 Rings - YAML.php.old
Andrew Kurn - The Philosopher
Preben Peppe Guldber - vim Master - yaml.syn
Why - The Lucky Stiff - YAML.rb
YAML: A New Language for Data
Resources: www.yaml.org
•
•
•
•
•
•
http://www.yaml.org
http://www.yaml.org/spec
[email protected]
http://wiki.yaml.org
perldoc YAML
http://wwysh.yaml.org
YAML: A New Language for Data
THE END
...