Processing XML with Java

Elliotte Rusty Harold

Copyright 2001, 2002 Elliotte Rusty Harold

Welcome to Processing XML with Java, a complete tutorial about writing Java programs that read and write XML documents. This is an approximately 1000 page book that will be published by Addison-Wesley in November 2002.

However, as I'm writing it, I'm going to be posting the entire book here at Cafe con Leche. You can read every word of it, more or less as I write it. Right now the first draft is available. This is composed of seventeen chapters and two appendixes, organized as follows:

  • Preface
  • Acknowledgements
  • 1 XML for Data
  • 2 XML Protocols
  • 3 Writing XML with Java
  • 4 Converting Flat Files to XML
  • 5 Reading XML
  • 6 SAX
  • 7 The XMLReader Interface
  • 8 SAX Filters
  • 9 The Document Object Model
  • 10 Creating New XML Documents with DOM
  • 11 The Document Object Model Core
  • 12 The DOM Traversal Module
  • 13 Output from DOM
  • 14 JDOM
  • 15 The JDOM Model
  • 16 XPath
  • 17 XSLT
  • A XML APIs Quick Reference
  • B SOAP Schemas
  • Recommended Reading

Your commentary and feedback is much desired, both on major issues (e.g. "Why don't you cover JAXB?") and minor ones ("Cat is misspelled in the first sentence of page 42.") Please send all feedback directly to me at elharo@metalab.unc.edu.

Prerequisites

This is not an introductory book. It assumes that you're completely familiar with Java including objects, classes, polymorphism, I/O, network programming, and more. It also assumes that you're familiar with XML at about the level of the XML Bible, 2nd Edition. In many ways this book picks up where that one left off.

Copying these Files

For the moment, I'd like to ask that you not mirror these files on your own servers. They'll be changing very rapidly, and I don't want to deal with questions based on out of date copies.

If you want to save a copy on your local hard drive for your own use, feel free, However, you should remember that these files change very fast. If at all possible, you should refer to the official site at http://www.cafeconleche.org/books/xmljava/ instead. Please don't pass out copies of your own copies to anyone else. Instead refer your friends and colleagues to this web site.

For now the files are available as you see them here in HTML. Once I get the stylesheets more debugged, I'll probably make a bundle of the full files available in Docbook, PDF, and other HTML formats as well. However, right now I'm concentrating on writing. Some of these tasks are going to have to wait until the book is more complete and changing less quickly.

Colophon

This entire book is being written in XML from start to finish. The specific XML application used is DocBook 4.2.0. I use the jEdit text editor on Windows and Linux to write. XInclude is used to merge the individual chapters and examples together. Michael Kay's SAXON XSLT processor and Norm Walsh's XSL stylesheets for Docbook 1.52.0 produce the HTML and XSL-FO output. I use FOP 0.20.4 to convert the XSL-FO files to PDFs.

Copyright 2001, 2002 Elliotte Rusty Harold
elharo@metalab.unc.edu
Last Modified October 14, 2002


Appendix A. XML APIs Quick Reference
Prev Next

Appendix A. XML APIs Quick Reference

Table of Contents

SAX
org.xml.sax
org.xml.sax.ext
org.xml.sax.helpers
DOM
The DOM Data Model
org.w3c.dom
org.w3c.dom.traversal
JAXP
javax.xml.parsers
TrAX
javax.xml.transform
javax.xml.transform.stream
javax.xml.transform.dom
javax.xml.transform.sax
JDOM Quick Reference
org.jdom
org.jdom.filter
org.jdom.input
org.jdom.output
org.jdom.transform
org.jdom.xpath
XMLPULL
org.xmlpull.v1

This appendix provides complete signatures for the documented, public packages, classes, methods, and fields in the various APIs you might use to process XML with Java, including:

  • SAX 2.0

  • DOM Level 2

  • JDOM

  • JAXP 1.1

  • TrAX

  • XMLPULL 1.0.8

SAX

SAX, the Simple API for XML, is an event driven API that models the parser and the client application receiving information from the parser, rather than modeling the document. It’s suitable for documents that are fairly local; that is, where processing one piece of the document does not depend heavily on content from other pieces of the document. It is extremely fast, and far and away the most memory efficient of the various XML APIs. It is the only API that is easily able to handle documents in the multi-gigabyte range.

SAX is divided into three packages:

  • org.xml.sax, the core classes and interfaces used by all SAX parsers and by most SAX programs

  • org.xml.sax.helpers: factory classes for loading instances of the core SAX interfaces, as well as some implementations of those interfaces

  • org.xml.sax.ext: optional extensions to SAX not all parsers provide

org.xml.sax

The org.xml.sax package contains the core classes and interfaces that represent the parser, the client application, the input document, and more. It also contains a number of classes from SAX1 that can be safely ignored in modern applications.

AttributeList

The AttributeList interface was used in SAX1 to pass attribute sets to the startElement() method of DocumentHandler. It is now deprecated and should not be used.

package org.xml.sax;

public interface AttributeList  {

  public int    getLength();
  public String getName(int i);
  public String getType(int i);
  public String getValue(int i);
  public String getType(String name);
  public String getValue(String name);

}

Attributes

The Attributes interface is used to pass attribute sets to the startElement() method of ContentHandler Although list-like, indexed access is supported for convenience, there is absolutely no guarantee that the attributes will be stored in the list in the order they appear in the start-tag. Indeed, in practice it’s more likely than not that they aren’t. If the attribute is declared to have any type other than CDATA (e.g., ID, NMTOKENS, enumerated, etc.) and the parser has read the DTD, then the reported value is normalized; that is, leading and trailing whitespace is trimmed and runs of whitespace are converted to single space. The unnormalized value will not be available. Undeclared attributes are not normalized.

package org.xml.sax;

public interface Attributes  {

  public int    getLength();
  public String getURI(int index);
  public String getLocalName(int index);
  public String getQName(int index);
  public String getType(int index);
  public String getValue(int index);
  public int    getIndex(String uri, String localName);
  public int    getIndex(String qualifiedName);
  public String getType(String uri, String localName);
  public String getType(String qualifiedName);
  public String getValue(String uri, String localName);
  public String getValue(String qualifiedName);

}

ContentHandler

ContentHandler is the basic callback interface a client application implements in order to receive information from the parser. Client applications register an instance of this interface with the parser using the setContentHandler() method in XMLReader. The parser will invoke the methods in that object in the same order that the corresponding markup and text appears in the original document.

Note

This interface has no relation to the moribund java.net.ContentHandler. However, the name does conflict, so don't import java.net.* in a class that uses org.xml.sax.ContentHandler. Just import the classes you actually use instead.

package org.xml.sax;

public interface ContentHandler  {

  public void setDocumentLocator(Locator locator);
  public void startDocument()
   throws SAXException;
  public void endDocument()
   throws SAXException;
  public void startPrefixMapping(String prefix, String uri) throws SAXException;
  public void endPrefixMapping(String prefix) throws SAXException;
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException;
  public void endElement(String namespaceURI, String localName,
   String qualifiedName) throws SAXException;
  public void characters(char[] text, int start, int length)
   throws SAXException;
  public void ignorableWhitespace(char[] space, int start, int length)
   throws SAXException;
  public void processingInstruction(String target, String data) throws SAXException;
  public void skippedEntity(String name) throws SAXException;

}

DTDHandler

DTDHandler is a callback interface a client application can implement and register with the XMLReader using the setDTDHandler() method if it wishes to receive information from the parser about notations and unparsed entities declared in the DTD. These methods will be called sometime after startDocument() and sometime before the startElement() method for the root element. You'll generally store the information in some data structure such as a Map so you can refer back to it when you an encounter an attribute with type NOTATION or ENTITY.

package org.xml.sax;

public interface DTDHandler  {

  public void notationDecl(String name, String publicID,
   String systemID) throws SAXException;
  public void unparsedEntityDecl(String name, String publicID,
   String systemID, String notationName) throws SAXException;

}

DocumentHandler

DocumentHandler is the SAX1 equivalent of ContentHandler. It is now deprecated and should not be used.

package org.xml.sax;

public interface DocumentHandler  {

  public void setDocumentLocator(Locator locator);
  public void startDocument() throws SAXException;
  public void endDocument() throws SAXException;
  public void startElement(String name, AttributeList atts)
   throws SAXException;
  public void endElement(String name) throws SAXException;
  public void characters(char[] text, int start, int length)
   throws SAXException;
  public void ignorableWhitespace(char[] space, int start, int length)
   throws SAXException;
  public void processingInstruction(String target, String data)
   throws SAXException;

}

EntityResolver

A client program can implement the EntityResolver interface to substitute different sources for entities referenced in the main document by public or system ID. A specific instance of EntityResolver is registered with the parser by passing it to the setEntityResolver() method in XMLReader. If resolveEntity() cannot locate a requested entity, it should return null to tell the parser to use its default entity resolution mechanism.

package org.xml.sax;

public interface EntityResolver  {

  public InputSource resolveEntity(String publicID,
   String systemID) throws SAXException, IOException;

}

ErrorHandler

ErrorHandler is a callback interface parsers use to report the three levels of errors that may be encountered while parsing an XML document: a fatal well-formedness error, a non-fatal error such as a validity violation, and a warning. Parsers normally stop parsing the current document after reporting a fatal error. That is, the parser calls endDocument() and the parse() method returns. If you want the same behavior from a non-fatal error, you should rethrow the exception from inside the error() method. You install a specific instance of this interface by passing it to the setErrorHandler() method in XMLReader.

package org.xml.sax;

public interface ErrorHandler  {

  public void warning(SAXParseException ex) throws SAXException;
  public void error(SAXParseException ex) throws SAXException;
  public void fatalError(SAXParseException ex)
   throws SAXException;

}

HandlerBase

HandlerBase is the SAX1 equivalent of DefaultHandler, that is an adapter class that provides do-nothing implementations of the SAX1 callback interfaces. It is now deprecated and should not be used.

package org.xml.sax;

public class HandlerBase implements EntityResolver,
 DTDHandler, DocumentHandler, ErrorHandler {

  public HandlerBase();

  // EntityResolver methods
  public InputSource resolveEntity(String publicID,
   String systemID) throws SAXException;

  // DTDHandler methods
  public void notationDecl(String name, String publicID,
   String systemID);
  public void unparsedEntityDecl(String name, String publicID,
   String systemID, String notationName);

  // DocumentHandler methods
  public void setDocumentLocator(Locator locator);
  public void startDocument() throws SAXException;
  public void endDocument() throws SAXException;
  public void startElement(String name, AttributeList attributes)
   throws SAXException;
  public void endElement(String name) throws SAXException;
  public void characters(char[] text, int start, int length)
   throws SAXException;
  public void ignorableWhitespace(char[] space, int start, int length)
   throws SAXException;
  public void processingInstruction(String target, String data)
   throws SAXException;

  // ErrorHandler methods
  public void warning(SAXParseException ex) throws SAXException;
  public void error(SAXParseException ex) throws SAXException;
  public void fatalError(SAXParseException ex)
   throws SAXException;

}

InputSource

InputSource is a wrapper for all the different things that may contain an XML document including URLs, InputStreams, and Readers. An InputSource should be configured with a system ID URL and either an InputStream or a Reader.

package org.xml.sax;

public class InputSource  {

  public InputSource();
  public InputSource(String systemID);
  public InputSource(InputStream byteStream);
  public InputSource(Reader characterStream);

  public void        setPublicId(String publicID);
  public String      getPublicId();
  public void        setSystemId(String url);
  public String      getSystemId();
  public void        setByteStream(InputStream in);
  public InputStream getByteStream();
  public void        setEncoding(String encoding);
  public String      getEncoding();
  public void        setCharacterStream(Reader in);
  public Reader      getCharacterStream();

}

You should always set the system ID for an InputSource. If nothing else is set, then the parser will read the document from that URL. You may also set either the byte stream or the character stream, in which case that will be used to read the document. If you set both the byte stream and the character stream, then the parser will read from the character stream.

Locator

Many parsers pass a Locator object to the setLocator() method of ContentHandler before they call startDocument(). If a parser does this (and it’s not required to), client applications can use that Locator object to determine in which entity at which line and column a particular event occurs. The location information returned is not guaranteed to be perfectly accurate, but it’s normally close enough to help in debugging.

package org.xml.sax;

public interface Locator  {

  public String getPublicId();
  public String getSystemId();
  public int    getLineNumber();
  public int    getColumnNumber();

}

Parser

Parser is the SAX1 class that represents an XML parser. Modern programs should use SAX2’s XMLReader instead.

package org.xml.sax;

public interface Parser  {

  public void setLocale(Locale locale) throws SAXException;
  public void setEntityResolver(EntityResolver resolver);
  public void setDTDHandler(DTDHandler handler);
  public void setDocumentHandler(DocumentHandler handler);
  public void setErrorHandler(ErrorHandler handler);

  public void parse(InputSource source)
   throws SAXException, IOException;
  public void parse(String systemID)
   throws SAXException, IOException;

}

XMLFilter

XMLFilter is an XMLReader that receives its content from another XMLReader rather than directly from a parsed document. Thus it has the opportunity to modify, log, replace, or otherwise manipulate the content the parser sends. However, most of the time it is much easier to write a filter by subclassing the org.xml.sax.helpers.XMLFilterImpl class rather than by implementing this interface directly.

package org.xml.sax;

public interface XMLFilter implements XMLReader {

  public void      setParent(XMLReader parent);
  public XMLReader getParent();

}

XMLReader

XMLReader is SAX2’s representation of an XML parser. It has methods to get and set features and properties, to configure the parser with the various handler classes, and to parse documents from an InputSource or a URL.

package org.xml.sax;

public interface XMLReader  {

  public boolean getFeature(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public void    setFeature(String name, boolean value)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public Object  getProperty(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public void    setProperty(String name, Object value)
   throws SAXNotRecognizedException, SAXNotSupportedException;

  public void         setEntityResolver(EntityResolver resolver);
  public EntityResolver getEntityResolver();
  public void           setDTDHandler(DTDHandler handler);
  public DTDHandler     getDTDHandler();
  public void          setContentHandler(ContentHandler handler);
  public ContentHandler getContentHandler();
  public void           setErrorHandler(ErrorHandler handler);
  public ErrorHandler   getErrorHandler();

  public void parse(InputSource input)
   throws IOException, SAXException;
  public void parse(String systemID)
   throws IOException, SAXException;

}

Exceptions and Errors

All SAX exceptions are checked exceptions and all are derived from org.xml.sax.SAXException. In addition, a few methods that perform I/O can also throw an IOException.

SAXException

SAXException is the generic checked exception for just about anything that can go wrong while parsing an XML document except an I/O error. Often a more specific subclass of SAXException will be thrown instead, even if the method is only declared to throw SAXException. The SAXException class predates Java 1.4 by a couple of years, so it uses its own getException() method to report the nested exception that caused the SAXException (if such exists) rather than the getCause() method introduced in Java 1.4.

package org.xml.sax;

public class SAXException extends Exception {

  public SAXException(String message);
  public SAXException(Exception ex);
  public SAXException(String message, Exception ex);

  public String    getMessage();
  public Exception getException();
  public String    toString();

}
SAXNotRecognizedException

A SAXNotRecognizedException signals that the parser does not support the feature or property you’re trying to set; e.g. you're trying to turn on validation for a non-validating parser.

package org.xml.sax;

public class SAXNotRecognizedException extends SAXException {

  public SAXNotRecognizedException(String message);

}
SAXNotSupportedException

A SAXNotSupportedException indicates one of two things. Either the parser cannot set a feature or property to the requested value (e.g. you’re trying to set the value of the http://xml.org/sax/properties/lexical-handler property to an object that does not implement LexicalHandler) or it cannot set that feature or property at this point in time (e.g. you’re trying to turn on validation when the parser is halfway through the document.)

package org.xml.sax;

public class SAXNotSupportedException extends SAXException {

  public SAXNotSupportedException(String message);

}
SAXParseException

A SAXParseException normally indicates a well-formedness error. In ErrorHandler it is also used to indicate validity errors, other non-fatal errors, and warnings. It normally provides line and column numbers locating the position of the error in the XML document that caused the exception. However, these are sometimes set to -1 if the error occurs very early in the document or if the parser can't determine exactly where the problem is.

package org.xml.sax;

public class SAXParseException extends SAXException {

  public SAXParseException(String message, Locator locator);
  public SAXParseException(String message, Locator locator,
   Exception e);
  public SAXParseException(String message, String publicID,
   String systemID, int lineNumber, int columnNumber);
  public SAXParseException(String message, String publicID,
   String systemID, int lineNumber, int columnNumber,
   Exception e);

  public String getPublicId();
  public String getSystemId();
  public int    getLineNumber();
  public int    getColumnNumber();

}

org.xml.sax.ext

The org.xml.sax.ext package contains optional extensions to SAX that parsers are not required to implement. In practice, the major parsers do implement the DeclHandler and LexicalHandler callback interfaces from this package.

DeclHandler

DeclHandler is a callback interface client applications can implement and register with the parser in order to receive notification of element, attribute, and parsed entity declarations. Parsers are not required to provide this information, but most validating parsers do. If a LexicalHandler is also installed, then all calls to these methods occur in between the calls to startDTD() and endDTD().

A DeclHandler is installed in an XMLReader as the value of the http://xml.org/sax/properties/declaration-handler property with the setProperty() method. Parsers that do not provide declaration events will throw a SAXNotRecognizedException when you do this.

package org.xml.sax.ext;

public interface DeclHandler  {

  public void elementDecl(String name, String model)
   throws SAXException;
  public void attributeDecl(String elementName,
   String attributeName, String type, String valueDefault,
   String value) throws SAXException;
  public void internalEntityDecl(String name, String value)
   throws SAXException;
  public void externalEntityDecl(String name, String publicID,
   String systemID) throws SAXException;

}

LexicalHandler

LexicalHandler is a callback interface client applications can implement and register with the parser in order to receive notification of lexical information such as comments, parsed entities, and CDATA section boundaries. Parsers are not required to provide lexical information, though most do. In normal usage, this interface is not common. Excessive reliance on it often indicates a misdesigned XML application.

A LexicalHandler is installed in an XMLReader as the value of the http://xml.org/sax/properties/lexical-handler property with the setProperty() method. Parsers that do not provide lexical events will throw a SAXNotRecognizedException when you do this.

package org.xml.sax.ext;

public interface LexicalHandler  {

  public void startDTD(String name, String publicID,
   String systemID) throws SAXException;
  public void endDTD() throws SAXException;
  public void startEntity(String name) throws SAXException;
  public void endEntity(String name) throws SAXException;
  public void startCDATA() throws SAXException;
  public void endCDATA() throws SAXException;
  public void comment(char[] text, int start, int length)
   throws SAXException;

}

org.xml.sax.helpers

The org.xml.sax.helpers package contains classes that may assist with many SAX tasks, but are not necessarily required. The ones client programmers will find most useful are the factory classes for locating XMLReaders, but there are also classes that provide minimal implementations of several SAX interfaces such as Attributes and that can be useful when writing filters.

AttributeListImpl

The legacy AttributeListImpl class is a SAX1 class that parser vendors may (or may not) use to implement the AttributesList interface. There’s little to no need for this in SAX2.

package org.xml.sax.helpers;

public class AttributeListImpl implements AttributeList {

  public AttributeListImpl();
  public AttributeListImpl(AttributeList atts);

  public int    getLength();
  public String getName(int i);
  public String getType(int i);
  public String getValue(int i);
  public String getType(String name);
  public String getValue(String name);

  public void   setAttributeList(AttributeList atts);
  public void   addAttribute(String name, String type,
   String value);
  public void   removeAttribute(String name);
  public void   clear();

}

AttributesImpl

AttributesImpl is a class that parser vendors may (or may not) use to implement the Attributes interface. It can also be useful for taking persistent snapshots of Attributes objects and modifying Attributes lists in filters.

package org.xml.sax.helpers;

public class AttributesImpl implements Attributes {

  public AttributesImpl();
  public AttributesImpl(Attributes atts);

  public int    getLength();
  public String getURI(int index);
  public String getLocalName(int index);
  public String getQName(int index);
  public String getType(int index);
  public String getValue(int index);
  public void   removeAttribute(int index);
  public void   setURI(int index, String uri);
  public void   setLocalName(int index, String localName);
  public void   setQName(int index, String qualifiedName);
  public void   setType(int index, String type);
  public void   setValue(int index, String value);

  public int    getIndex(String uri, String localName);
  public int    getIndex(String qualifiedName);
  public String getType(String uri, String localName);
  public String getType(String qualifiedName);
  public String getValue(String uri, String localName);
  public String getValue(String qualifiedName);

  public void   clear();
  public void   setAttributes(Attributes atts);
  public void   addAttribute(String uri, String localName,
   String qualifiedName, String type, String value);
  public void   setAttribute(int index, String uri,
   String localName, String qualifiedName, String type,
   String value);

}

DefaultHandler

DefaultHandler is an adapter class that implements the four required SAX2 callback interfaces—EntityResolver, DTDHandler, ContentHandler, and ErrorHandler— with do-nothing implementations of all their methods. It’s often more convenient to subclass this class rather than implement those interfaces directly.

package org.xml.sax.helpers;

public class DefaultHandler implements EntityResolver,
 DTDHandler, ContentHandler, ErrorHandler {

  public DefaultHandler();

  // EntityResolver methods
  public InputSource resolveEntity(String publicID,
   String systemID) throws SAXException;

  // DTDHandler methods
  public void notationDecl(String name, String publicID,
   String systemID) throws SAXException;
  public void unparsedEntityDecl(String name, String publicID,
   String systemID, String notationName) throws SAXException;

  // ContentHandler methods
  public void setDocumentLocator(Locator locator);
  public void startDocument() throws SAXException;
  public void endDocument() throws SAXException;
  public void startPrefixMapping(String prefix, String uri)
   throws SAXException;
  public void endPrefixMapping(String prefix)
   throws SAXException;
  public void startElement(String uri, String localName,
   String qualifiedName, Attributes attributes)
   throws SAXException;
  public void endElement(String uri, String localName,
   String qualifiedName) throws SAXException;
  public void characters(char[] text, int start, int length)
   throws SAXException;
  public void ignorableWhitespace(char[] space, int start, int length)
   throws SAXException;
  public void processingInstruction(String target, String data)
   throws SAXException;
  public void skippedEntity(String name) throws SAXException;

  // ErrorHandler methods
  public void warning(SAXParseException e) throws SAXException;
  public void error(SAXParseException e) throws SAXException;
  public void fatalError(SAXParseException e) throws SAXException;

}

LocatorImpl

LocatorImpl is a class that parser vendors may take advantage of to help them provide the optional Locator interface. There’s little reason for most developers to use it, though it may occasionally be useful if you want to provide pseudo-location information from a filter or non-parser-based XMLReader or you want to take a persistent snapshot of location information at some point in the parse process.

package org.xml.sax.helpers;

public class LocatorImpl implements Locator {

  public LocatorImpl();
  public LocatorImpl(Locator locator);

  public String getPublicId();
  public String getSystemId();
  public int    getLineNumber();
  public int    getColumnNumber();

  public void setPublicId(String publicID);
  public void setSystemId(String systemID);
  public void setLineNumber(int lineNumber);
  public void setColumnNumber(int columnNumber);

}

NamespaceSupport

The NamespaceSupport class provides a stack for storing namespace prefix-URI mappings. It’s useful when a program needs to resolve namespace prefixes that occur in attribute values and element content. You can reuse a NamespaceSupport object for multiple documents. However, if you do so, you need to call reset() in between documents (typically in startDocument()).

package org.xml.sax.helpers;

public class NamespaceSupport  {

  public static final String XMLNS;

  public NamespaceSupport();

  public void        reset();
  public void        pushContext();
  public void        popContext();
  public boolean     declarePrefix(String prefix, String uri);
  public String[]    processName(String qualifiedName,
   String parts, boolean isAttribute);

  public String      getURI(String prefix);
  public Enumeration getPrefixes();
  public String      getPrefix(String uri);
  public Enumeration getPrefixes(String uri);
  public Enumeration getDeclaredPrefixes();

}

ParserAdapter

ParserAdapter uses the adapter design pattern to convert SAX1 Parser objects into SAX2 XMLReader objects. Given the large number of high-quality, native SAX2 parsers that are now available, there’s little reason to use this class anymore.

package org.xml.sax.helpers;

public class ParserAdapter implements XMLReader, DocumentHandler {

  public ParserAdapter() throws SAXException;
  public ParserAdapter(Parser parser);

  // XMLReader methods
  public void setFeature(String name, boolean state)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public boolean getFeature(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public void setProperty(String name, Object value)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public Object getProperty(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public void setEntityResolver(EntityResolver resolver);
  public EntityResolver getEntityResolver();
  public void setDTDHandler(DTDHandler handler);
  public DTDHandler getDTDHandler();
  public void setContentHandler(ContentHandler handler);
  public ContentHandler getContentHandler();
  public void setErrorHandler(ErrorHandler handler);
  public ErrorHandler getErrorHandler();

  public void parse(String systemID)
   throws IOException, SAXException;
  public void parse(InputSource input)
   throws IOException, SAXException;

  // DocumentHandler methods
  public void setDocumentLocator(Locator locator);
  public void startDocument() throws SAXException;
  public void endDocument() throws SAXException;
  public void startElement(String qualifiedName,
   AttributeList qAtts) throws SAXException;
  public void endElement(String qualifiedName)
   throws SAXException;
  public void characters(char[] text, int start, int length)
   throws SAXException;
  public void ignorableWhitespace(char[] space, int start, int length)
   throws SAXException;
  public void processingInstruction(String target, String data)
   throws SAXException;

}

ParserFactory

ParserFactory is a factory class used to create implementation specific instances of the SAX1 Parser class. The default implementation is determined by the org.xml.sax.parser Java system property. ParserFactory is now deprecated like the rest of SAX1, and should not be used.

package org.xml.sax.helpers;

public class ParserFactory  {

  public static Parser makeParser()
   throws ClassNotFoundException, IllegalAccessException,
          InstantiationException, NullPointerException,
          ClassCastException;
  public static Parser makeParser(String className)
   throws ClassNotFoundException, IllegalAccessException,
          InstantiationException, ClassCastException;

}

XMLFilterImpl

Although there is a more generic XMLFilter interface, XMLFilterImpl is the class you should actually subclass to implement SAX filters. The parent XMLReader is the object from which the filter will receive its information. By default, all the methods in this class simply delegate to the parent’s equivalent methods or objects. However, you can override any or all of them to perform the filtering.

package org.xml.sax.helpers;

public class XMLFilterImpl
 implements XMLFilter, EntityResolver, DTDHandler,
            ContentHandler, ErrorHandler {

  public XMLFilterImpl();
  public XMLFilterImpl(XMLReader parent);

  public void      setParent(XMLReader parent);
  public XMLReader getParent();

  // XMLReader methods
  public void           setFeature(String name, boolean state)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public boolean        getFeature(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public void           setProperty(String name, Object value)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public Object         getProperty(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public void          setEntityResolver(EntityResolver resolver);
  public EntityResolver getEntityResolver();
  public void           setDTDHandler(DTDHandler handler);
  public DTDHandler     getDTDHandler();
  public void          setContentHandler(ContentHandler handler);
  public ContentHandler getContentHandler();
  public void           setErrorHandler(ErrorHandler handler);
  public ErrorHandler   getErrorHandler();

  public void           parse(InputSource input)
   throws SAXException, IOException;
  public void           parse(String systemID)
   throws SAXException, IOException;

  // EntityResolver methods
  public InputSource resolveEntity(String publicID,
   String systemID) throws SAXException, IOException;

  // DTDHandler methods
  public void notationDecl(String name, String publicID,
   String systemID) throws SAXException;
  public void unparsedEntityDecl(String name, String publicID,
   String systemID, String notationName) throws SAXException;

  // ContentHandler methods
  public void setDocumentLocator(Locator locator);
  public void startDocument() throws SAXException;
  public void endDocument() throws SAXException;
  public void startPrefixMapping(String prefix, String uri)
   throws SAXException;
  public void endPrefixMapping(String prefix)
   throws SAXException;
  public void startElement(String uri, String localName,
   String qualifiedName, Attributes atts) throws SAXException;
  public void endElement(String uri, String localName,
   String qualifiedName) throws SAXException;
  public void characters(char[] text, int start, int length)
   throws SAXException;
  public void ignorableWhitespace(char[] space, int start, int length)
   throws SAXException;
  public void processingInstruction(String target, String data)
   throws SAXException;
  public void skippedEntity(String name) throws SAXException;

  // ErrorHandler methods
  public void warning(SAXParseException e) throws SAXException;
  public void error(SAXParseException e) throws SAXException;
  public void fatalError(SAXParseException e)
   throws SAXException;

}

XMLReaderAdapter

XMLReaderAdapter uses the adapter design pattern to convert SAX2 XMLReader objects into SAX1 Parser objects for compatibility with legacy code.

package org.xml.sax.helpers;

public class XMLReaderAdapter implements Parser, ContentHandler {

  public XMLReaderAdapter() throws SAXException;
  public XMLReaderAdapter(XMLReader xmlReader);

  // Parser methods
  public void setLocale(Locale locale) throws SAXException;
  public void setEntityResolver(EntityResolver resolver);
  public void setDTDHandler(DTDHandler handler);
  public void setDocumentHandler(DocumentHandler handler);
  public void setErrorHandler(ErrorHandler handler);

  public void parse(String systemID) throws IOException, SAXException;
  public void parse(InputSource input) throws IOException, SAXException;

  // ContentHandler methods
  public void setDocumentLocator(Locator locator);
  public void startDocument() throws SAXException;
  public void endDocument() throws SAXException;
  public void startPrefixMapping(String prefix, String uri);
  public void endPrefixMapping(String prefix);
  public void startElement(String uri, String localName,
   String qualifiedName, Attributes atts) throws SAXException;
  public void endElement(String uri, String localName,
   String qualifiedName) throws SAXException;
  public void characters(char[] text, int start, int length)
   throws SAXException;
  public void ignorableWhitespace(char[] space, int start, int length)
   throws SAXException;
  public void processingInstruction(String target, String data)
   throws SAXException;
  public void skippedEntity(String name) throws SAXException;

}

XMLReaderFactory

XMLReaderFactory is a factory class used to create implementation specific instances of XMLReader. The default implementation is determined by the org.xml.sax.driver Java system property.

package org.xml.sax.helpers;

public final class XMLReaderFactory  {

  public static XMLReader createXMLReader() throws SAXException;
  public static XMLReader createXMLReader(String className)
   throws SAXException;

}
Prev | Up | Next
Summary | Home | DOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 03, 2002
 | Up To Cafe con Leche |


DOM
Prev Appendix A. XML APIs Quick Reference Next

DOM

The Document Object Model defines a tree-based representation of XML documents. The org.w3c.dom package contains the basic node classes that represent the different components that make up the tree. The org.w3c.dom.traversal package includes some useful utility classes for navigating, searching, and querying the tree.

DOM Level 2, the version which is described here, is incomplete. It does not define how a DOMImplementation is loaded, how a document is parsed, or how a document is serialized. For the moment, JAXP provides a stopgap solution. Eventually, DOM Level 3 will fill in these holes. However, since DOM Level 3 was far from complete at the time of this writing, this appendix covers DOM level 2 exclusively.

The DOM Data Model

Table A.1 summarizes the DOM data model with the name, value, parent, and possible children for each kind of node.

Table A.1. Node properties

Node type | name | value | parent | children
Document | #document | null | null | Comment, processing instruction, zero or one document type, one element
DocumentType | Root element name specified by the DOCTYPE declaration | null | Document | none
Element | prefixed name | null | Element, Document, or Document fragment | Comment, Processing Instruction, Text, Element, Entity reference, CDATA section
Text | #text | text of the node | Element, Attr, Entity, or Entity reference | none
Attr | prefixed name | normalized attribute value | Element | Text, Entity reference
Comment | #comment | text of comment | Element, Document, or Document fragment | none
Processing Instruction | target | data | Element, Document, or Document fragment | none
Entity Reference | name | null | Element or Document Fragment | Comment, Processing Instruction, Text, Element, Entity reference, CDATA section
Entity | entity name | null | null | Comment, Processing Instruction, Text, Element, Entity Reference, CDATA section
CDATA section | #cdata-section | text of the section | Element, Entity, or Entity reference | none
Notation | notation name | null | null | none
Document fragment | #document-fragment | null | null | Comment, Processing Instruction, Text, Element, Entity reference, CDATA section

One thing to keep in mind is the parts of the XML document that are not exposed in this data model:

  • The XML declaration, including the version, standalone, and encoding declarations. These will be added as properties of the document node in DOM3, but they are not provided by current parsers.

  • Most information from the DTD and/or schema is not provided including element and attribute types and content models. DOM Level 3 will add some of this.

  • Any white space outside the root element.

  • Whether or not each character was provided by a character reference. Parsers may provide information about entity references, but are not required to do so.

A DOM program cannot manipulate any of these constructs. It cannot, for example, read in an XML document, then write it out again in the same encoding the original document used because it doesn’t know what encoding the original document used. It cannot treat $var differently than &#x24;var because it doesn’t know which was originally written.

org.w3c.dom

The org.w3c.dom package contains the core interfaces that are used to form DOM documents. Node is the common superinterface all these node types share. In addition, this package contains a few data structures used to hold collections of DOM nodes and one exception class.

Attr

The Attr interface represents an attribute node. Its node properties are defined as follows:

node name | The full name of the attribute, including a prefix and a colon if the attribute is in a namespace
node value | The attribute’s normalized value
local name | The local part of the attribute’s name
namespace URI | The namespace URI of the attribute or null if the attribute does not have a prefix
namespace prefix | The namespace URI of the attribute or null if the attribute is not in a namespace

Furthermore, Attr objects are not part of the tree. They have neither parents nor siblings. getParentNode(), getPreviousSibling(), and getNextSibling() all return null when invoked on an Attr object. Attr objects do have children (Text and EntityReference objects) but it's generally best to ignore this fact, and just use the getValue() method to read the value of an attribute.

package org.w3c.dom;

public interface Attr extends Node {

  public String  getName();
  public boolean getSpecified();
  public String  getValue();
  public void    setValue(String value) throws DOMException;
  public Element getOwnerElement();

}

CDATASection

The CDATASection interface represents a CDATA section. DOM parsers are not required to use this interface to report CDATA sections. They may just use Text objects to report the content of CDATA sections. Do not write code that depends on recognizing CDATA sections in text. The node properties of CDATASection are defined as follows:

node name | #cdata-section
node value | The text of the CDATA section
local name | null
namespace URI | null
namespace prefix | null
package org.w3c.dom;

public interface CDATASection extends Text {

}

CharacterData

The CharacterData interface is the generic superinterface for those nodes composed of plain text: Comment, Text, and CDATASection. All actual instances of CharacterData should be instances of one of these subinterfaces. The node properties depend on the specific subinterface.

package org.w3c.dom;

public interface CharacterData extends Node {

  public String getData() throws DOMException;
  public void   setData(String data) throws DOMException;
  public int    getLength();
  public String substringData(int offset, int count)
   throws DOMException;
  public void   appendData(String s) throws DOMException;
  public void   insertData(int offset, String s)
   throws DOMException;
  public void   deleteData(int offset, int count)
   throws DOMException;
  public void   replaceData(int offset, int count, String s)
   throws DOMException;

}

Comment

The Comment interface represents a comment node. All its methods are inherited from the CharacterData and Node superinterfaces. Its node properties are defined as follows:

node name | #comment
node value | The text of the comment, not including <-- and -->
local name | null
namespace URI | null
namespace prefix | null
package org.w3c.dom;

public interface Comment extends CharacterData {

}

Document

The Document interface represents the root node of the tree. It also serves as an abstract factory to create the other kinds of nodes (element, attribute, comment, etc.) which will be stored in the tree. Its node properties are defined as follows:

node name | #document
node value | null
local name | null
namespace URI | null
namespace prefix | null
package org.w3c.dom;

public interface Document extends Node {

  public DocumentType          getDoctype();
  public DOMImplementation     getImplementation();
  public Element               getDocumentElement();

  public Element               createElement(String tagName)
   throws DOMException;
  public Element               createElementNS(
   String namespaceURI, String qualifiedName) throws DOMException;
  public Attr                  createAttribute(String name)
   throws DOMException;
  public Attr                  createAttributeNS(
   String namespaceURI, String qualifiedName) throws DOMException;
  public DocumentFragment      createDocumentFragment();
  public Text                  createTextNode(String data);
  public Comment               createComment(String data);
  public CDATASection          createCDATASection(String data)
   throws DOMException;
  public ProcessingInstruction createProcessingInstruction(
   String target, String data) throws DOMException;
  public EntityReference       createEntityReference(String name)
   throws DOMException;

  public NodeList getElementsByTagName(String tagName);
  public Node     importNode(Node importedNode, boolean deep)
   throws DOMException;
  public NodeList getElementsByTagNameNS(String namespaceURI,
   String localName);
  public Element  getElementById(String id);

}

DocumentFragment

The DocumentFragment interface is used to hold lists of element, text, comment, CDATA section, and processing instruction nodes when those nodes do not have a parent. It’s convenient for cutting and pasting or inserting and moving fragments of an XML document that that don’t necessarily contain a single element. Its node properties are defined as follows:

node name | #document-fragment
node value | null
local name | null
namespace URI | null
namespace prefix | null
package org.w3c.dom;

public interface DocumentFragment extends Node {

}

This interface is for advanced use only. DOM trees created by a parser won’t contain any DocumentFragment objects and adding a DocumentFragment to a Document actually adds the contents of the fragment instead.

DocumentType

The DocumentType interface represents a document type declaration. It contains the root element name it declares, the system ID and public ID for the external DTD subset, and the complete internal DTD subset as a String. It also contains lists of the notations and general entities declared in the DTD. Other than this it contains no information from the DTD. The node properties of a DocumentType object are defined as follows:

node name | declared root element name
node value | null
local name | null
namespace URI | null
namespace prefix | null
package org.w3c.dom;

public interface DocumentType extends Node {

  public String getName();
  public String getPublicId();
  public String getSystemId();
  public String getInternalSubset();

  public NamedNodeMap getEntities();
  public NamedNodeMap getNotations();

}

In DOM Level 2 the entire DocumentType object is read-only. No part of it can be modified. Furthermore a Document object’s DocumentType cannot be changed after the Document object is created. This restriction is lifted in DOM Level 3.

DOM does not provide any representation of the document type definition as distinguished from the document type declaration.

DOMImplementation

DOMImplementation is an abstract factory used to create new Document and DocumentType objects. The javax.xml.parsers.DocumentBuilder class can create new DOMImplementation objects.

package org.w3c.dom;

public interface DOMImplementation  {

  public DocumentType createDocumentType(String qualifiedName,
   String publicID, String systemID) throws DOMException;
  public Document     createDocument(String namespaceURI,
   String qualifiedName, DocumentType doctype) throws DOMException;

  public boolean hasFeature(String feature, String version);

}

Element

The Element interface represents an element node. The most important methods for this interface are inherited from the Node superinterface. Its node properties are defined as follows:

node name | The qualified name of the element, possibly including a prefix and a colon
node value | null
local name | The local part of the element name
namespace URI | The namespace URI of the element, or null if this element is not in a namespace
namespace prefix | The namespace prefix of the element, or null if this element is in the default namespace or no namespace at all
package org.w3c.dom;

public interface Element extends Node {

  public String   getTagName();
  public NodeList getElementsByTagNameNS(String namespaceURI,
   String localName);
  public NodeList getElementsByTagName(String name);

  public String  getAttribute(String name);
  public void    setAttribute(String name, String value)
   throws DOMException;
  public void    removeAttribute(String name)
   throws DOMException;
  public Attr    getAttributeNode(String name);
  public Attr    setAttributeNode(Attr newAttr)
   throws DOMException;
  public Attr    removeAttributeNode(Attr oldAttr)
   throws DOMException;
  public String  getAttributeNS(String namespaceURI,
   String localName);
  public void    setAttributeNS(String namespaceURI,
   String qualifiedName, String value) throws DOMException;
  public void    removeAttributeNS(String namespaceURI,
   String localName) throws DOMException;
  public Attr    getAttributeNodeNS(String namespaceURI,
   String localName);
  public Attr    setAttributeNodeNS(Attr newAttr)
   throws DOMException;
  public boolean hasAttribute(String name);
  public boolean hasAttributeNS(String namespaceURI,
   String localName);

}

Entity

The Entity interface represents an entity node. It does not appear directly in the tree. Instead an EntityReference node appears in the tree. The name of the EntityReference identifies a member of the document’s entities map, which is accessible through the DocumentType interface. If the Entity object represents a parsed entity, and the parser resolved the entity, then this node will have children representing its replacement text. However all aspects of the Entity object including all its children are read-only. They may not be modified or changed in any way.

package org.w3c.dom;

public interface Entity extends Node {

  public String getPublicId();
  public String getSystemId();
  public String getNotationName();

}

The node properties of Entity are defined as follows:

node name | The name of the entity
node value | null
local name | null
namespace URI | null
namespace prefix | null

Since Entity objects are not part of the tree, they have neither parents nor siblings. getParentNode(), getPreviousSibling(), and getNextSibling() all return null when invoked on an Entity object.

EntityReference

The EntityReference interface represents a parsed entity reference which appears in the document tree. Parsers are not required to use this class. Some parsers silently resolve all entity references to their replacement text. If a parser does not resolve external entity references, then it must include EntityReference objects instead, though the only information available from these objects will be the name. A parser that does resolve external entity references and chooses to include EntityReference objects anyway will also set the children of this node so as to the represent the entity’s replacement text. In this case, you can use the methods inherited from the Node superinterface to walk the entity’s tree. However, all these children and their descendants are completely read-only. You cannot change them in any way. If you need to modify them, you must first clone each of the EntityReference’s children, and replace the EntityReference with the cloned children.

package org.w3c.dom;

public interface EntityReference extends Node {

}

EntityReference objects are never used for the five predefined entity references (&lt;, &gt;, &amp;, &quot;, and &apos;,) or for character references such as &#xA0; or &#160;.

The node properties of EntityReference are defined as follows:

node name | The name of the entity
node value | null
local name | null
namespace URI | null
namespace prefix | null

NamedNodeMap

DOM uses NamedNodeMap data structures to hold unordered sets of attributes, notations, and entities. You can iterate through a map using the item() and getLength(). The first item in the map is at index 0. However, the particular order the implementation chooses is not significant or even reproducible.

package org.w3c.dom;

public interface NamedNodeMap  {

  public Node getNamedItem(String name);
  public Node setNamedItem(Node node) throws DOMException;
  public Node removeNamedItem(String name) throws DOMException;
  public Node item(int index);
  public int  getLength();
  public Node getNamedItemNS(String namespaceURI,
   String localName);
  public Node setNamedItemNS(Node node) throws DOMException;
  public Node removeNamedItemNS(String namespaceURI,
   String localName) throws DOMException;

}

NamedNodeMaps are live. That is, adding an item to the map or removing an item from the map will add it to or remove it from whatever construct produced the map in the first place.

Node

Node is the key superinterface for almost all the other classes in the org.w3c.dom package. It is the primary means by which you navigate, search, query, and occasionally even update an XML document with DOM.

package org.w3c.dom;

public interface Node  {

  // Node type constants
  public static final short ELEMENT_NODE;
  public static final short ATTRIBUTE_NODE;
  public static final short TEXT_NODE;
  public static final short CDATA_SECTION_NODE;
  public static final short ENTITY_REFERENCE_NODE;
  public static final short ENTITY_NODE;
  public static final short PROCESSING_INSTRUCTION_NODE;
  public static final short COMMENT_NODE;
  public static final short DOCUMENT_NODE;
  public static final short DOCUMENT_TYPE_NODE;
  public static final short DOCUMENT_FRAGMENT_NODE;
  public static final short NOTATION_NODE;

  // Basic getter methods
  public String   getNodeName();
  public String   getNodeValue() throws DOMException;
  public void     setNodeValue(String value) throws DOMException;
  public short    getNodeType();
  public String   getNamespaceURI();
  public String   getPrefix();
  public void     setPrefix(String prefix) throws DOMException;
  public String   getLocalName();

  // Navigation methods
  public Node     getParentNode();
  public boolean  hasChildNodes();
  public NodeList getChildNodes();
  public Node     getFirstChild();
  public Node     getLastChild();
  public Node     getPreviousSibling();
  public Node     getNextSibling();
  public Document getOwnerDocument();

  // Attribute methods
  public boolean      hasAttributes();
  public NamedNodeMap getAttributes();

  // Tree modification methods
  public Node     insertBefore(Node newChild, Node refChild)
   throws DOMException;
  public Node     replaceChild(Node newChild, Node oldChild)
   throws DOMException;
  public Node     removeChild(Node oldChild) throws DOMException;
  public Node     appendChild(Node newChild) throws DOMException;

  // Utility methods
  public Node     cloneNode(boolean deep);
  public void     normalize();
  public boolean  isSupported(String feature, String version);

}

NodeList

NodeList is the basic DOM list type. These are most commonly used for lists of children of an Element or Document. The index of the first item in the list is 0, like Java arrays.

The actual data structure used to implement the list can vary from implementation to implementation. However, one constant is that the lists are live. In other words, if a node is deleted or moved from its parent, then it is also deleted from all lists that were built from the children of that parent. Similarly, if a new node is added to some node, then it is also added to all lists that point to the children of that node.

package org.w3c.dom;

public interface NodeList  {

  public Node item(int index);
  public int  getLength();

}

Notation

The Notation interface represents a notation declared in the document’s DTD. It does not have a position in the tree. However, the complete list of notations in the document is accessible through the getNotations() method of the DocumentType interface. Both this list and the individual Notation objects are read-only.

package org.w3c.dom;

public interface Notation extends Node {

  public String getPublicId();
  public String getSystemId();

}

The node properties of Notation are defined as follows:

node name | notation name
node value | null
local name | null
namespace URI | null
namespace prefix | null

ProcessingInstruction

The ProcessingInstruction interface represents a processing instruction node. Its node properties are defined as follows:

node name | the target
node value | the data
local name | null
namespace URI | null
namespace prefix | null
package org.w3c.dom;

public interface ProcessingInstruction extends Node {

  public String getTarget();
  public String getData();
  public void   setData(String data) throws DOMException;

}

Text

The Text interface represents a text node. It can contain any characters that are legal in XML text including characters like < and & that may need to be escaped when the document is serialized. When a parser reads an XML document and builds a DOM tree, each Text object will contain the longest possible contiguous run of text. However, DOM does not maintain this constraint as the document is manipulated in memory. Its node properties are defined as follows:

node name | #text
node value | the text of the node
local name | null
namespace URI | null
namespace prefix | null

The Text interface only declares one method of its own, splitText(). Most of its functionality is inherited from the superinterfaces CharacterData and Node.

package org.w3c.dom;

public interface Text extends CharacterData {

  public Text splitText(int offset) throws DOMException;

}

Exceptions and Errors

DOM Level 2 defines only one exception class, DOMException. This is a runtime exception used for almost anything that can go wrong while constructing or manipulating a DOM Document The details are provided by a short field, code which is set to any of several named constants.

package org.w3c.dom;

public class DOMException extends RuntimeException {

  public short code;

  public static final short INDEX_SIZE_ERR;
  public static final short DOMSTRING_SIZE_ERR;
  public static final short HIERARCHY_REQUEST_ERR;
  public static final short WRONG_DOCUMENT_ERR;
  public static final short INVALID_CHARACTER_ERR;
  public static final short NO_DATA_ALLOWED_ERR;
  public static final short NO_MODIFICATION_ALLOWED_ERR;
  public static final short NOT_FOUND_ERR;
  public static final short NOT_SUPPORTED_ERR;
  public static final short INUSE_ATTRIBUTE_ERR;
  public static final short INVALID_STATE_ERR;
  public static final short SYNTAX_ERR;
  public static final short INVALID_MODIFICATION_ERR;
  public static final short NAMESPACE_ERR;
  public static final short INVALID_ACCESS_ERR;

  public DOMException(short code, String message);

}

org.w3c.dom.traversal

The DOM traversal API in the org.w3c.dom.traversal package provides some convenience classes for navigating and searching an XML document. The most useful aspects of this class are the ability to get lists and trees that contain the kinds of nodes that you’re interested in while ignoring everything else.

DocumentTraversal

DocumentTraversal is a factory interface for creating new NodeIterator and TreeWalker objects that present a filtered view of the content of an element or a document. (You can filter other kinds of nodes too, but there’s not a lot of point to this if they don’t have any children.)

In implementations that support the traversal API (which can be determined by invoking the hasFeature("Traversal", "2.0" ) method in the Document or DOMImplementation classes) all objects that implement Document also implement DocumentTraversal. That is, to create a DocumentTraversal object, just cast a Document to DocumentTraversal.

package org.w3c.dom.traversal;

public interface DocumentTraversal  {

  public NodeIterator createNodeIterator(Node root,
   int whatToShow, NodeFilter filter, boolean expandEntities)
   throws DOMException;
  public TreeWalker createTreeWalker(Node root, int whatToShow,
   NodeFilter filter, boolean expandEntities)
   throws DOMException;

}

NodeFilter

The NodeFilter interface is used by NodeIterators and TreeWalkers to determine which nodes are included in the view of the document they present to the client. Each node in the subtree will be passed to the filter’s acceptNode() method. This returns one of the three named constants NodeFilter.FILTER_ACCEPT (include the node), NodeFilter.FILTER_REJECT (do not include the node or any of its descendants when tree walking, do not include the node or but do include its descendants when iterating), or, NodeFilter.FILTER_SKIP (do not include the node but do include its children if they pass the filter individually).

In addition this class has thirteen named constants that can be combined with the bitwise operators and passed to createNodeIterator() and createTreeWalker() to specify which kinds of nodes should be included in their views.

package org.w3c.dom.traversal;

public interface NodeFilter  {

  public static final short FILTER_ACCEPT;
  public static final short FILTER_REJECT;
  public static final short FILTER_SKIP;

  public static final int SHOW_ALL;
  public static final int SHOW_ELEMENT;
  public static final int SHOW_ATTRIBUTE;
  public static final int SHOW_TEXT;
  public static final int SHOW_CDATA_SECTION;
  public static final int SHOW_ENTITY_REFERENCE;
  public static final int SHOW_ENTITY;
  public static final int SHOW_PROCESSING_INSTRUCTION;
  public static final int SHOW_COMMENT;
  public static final int SHOW_DOCUMENT;
  public static final int SHOW_DOCUMENT_TYPE;
  public static final int SHOW_DOCUMENT_FRAGMENT;
  public static final int SHOW_NOTATION;

  public short acceptNode(Node node);

}

NodeIterator

The NodeIterator interface presents a subset of nodes from the document as a list in document order. The list is live; that is, changes to the document are reflected in the list.

package org.w3c.dom.traversal;

public interface NodeIterator  {

  public Node       nextNode() throws DOMException;
  public Node       previousNode() throws DOMException;

  public Node       getRoot();
  public int        getWhatToShow();
  public NodeFilter getFilter();
  public boolean    getExpandEntityReferences();

  public void       detach();

}

TreeWalker

The TreeWalker interface presents a subset of nodes from the document as a tree. Walking the TreeWalker is much like walking a full Document or Element, except that many of the node’s descendants which you aren’t interested in can be filtered out so they don’t get in your way. The tree is live; that is, changes to the document are reflected in the tree.

package org.w3c.dom.traversal;

public interface TreeWalker  {

  public Node parentNode();
  public Node firstChild();
  public Node lastChild();
  public Node previousSibling();
  public Node nextSibling();
  public Node previousNode();
  public Node nextNode();

  public Node       getRoot();
  public int        getWhatToShow();
  public NodeFilter getFilter();
  public boolean    getExpandEntityReferences();
  public Node       getCurrentNode();
  public void       setCurrentNode(Node node)
   throws DOMException;

}
Prev | Up | Next
Appendix A. XML APIs Quick Reference | Home | JAXP
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 27, 2002


JAXP
Prev Appendix A. XML APIs Quick Reference Next

JAXP

JAXP, the Java API for XML Processing, is a fairly complete set of APIs for processing XML with Java. It is bundled with Java 1.4 and later, and available as a separate extension for Java 1.2 and later. (All pieces except TrAX should run in Java 1.1 and later.) Most current XML parsers for Java support JAXP including Crimson, Ælfred, Xerces, Piccolo, and the Oracle XML Parser for Java. JAXP incorporates SAX and DOM by reference. In addition it adds the Transformations API for XML (TrAX) and some factory classes for locating a parser and building new documents in memory. This section covers those factory classes, all of which are in the javax.xml.parsers package.

As well as adopting DOM and SAX into the core Java API, JAXP adds a few factory classes to fill some holes in these APIs and enable Java programmers to write completely parser-independent code.

javax.xml.parsers

The one major new invention in JAXP that was not based on previous standards was a series of abstract factory classes in the javax.xml.parsers package. These allow a Java program to obtain a DOM parser, a SAX1 Parser, or a DOMImplementation in a parser-independent fashion. The SAX1 factories are now obsolete, but the DOM factories are still quite useful.

DocumentBuilder

The DocumentBuilder class is an abstract factory used to create new DOM Document and DOMImplementation objects. As well as creating new instances from scratch, DocumentBuilder can also read a document from an InputStream, Reader, File, SAX InputSource, or a URI.

package javax.xml.parsers;

public abstract class DocumentBuilder  {

  protected DocumentBuilder();

  public Document          newDocument();
  public DOMImplementation getDOMImplementation();

  public Document parse(InputStream in)
   throws SAXException, IOException;
  public Document parse(InputStream in, String systemID)
   throws SAXException, IOException;
  public Document parse(String uri)
   throws SAXException, IOException;
  public Document parse(File f)
   throws SAXException, IOException;
  public Document parse(InputSource in)
   throws SAXException, IOException;

  public boolean  isNamespaceAware();
  public boolean  isValidating();
  public void     setEntityResolver(EntityResolver resolver);
  public void     setErrorHandler(ErrorHandler handler);

}

DocumentBuilderFactory

The DocumentBuilderFactory class is an abstract factory used to create new DocumentBuilder objects. You should always call setNamespaceAware(true) before calling newInstance(true).

package javax.xml.parsers;

public abstract class DocumentBuilderFactory  {

  protected DocumentBuilderFactory();

  public static DocumentBuilderFactory newInstance()
   throws FactoryConfigurationError;
  public DocumentBuilder newDocumentBuilder()
   throws ParserConfigurationException;

  public void    setNamespaceAware(boolean awareness);
  public void    setValidating(boolean validating);
  public void    setIgnoringElementContentWhitespace(
   boolean ignoreWhitespace);
  public void    setExpandEntityReferences(
   boolean expandEntities);
  public void    setIgnoringComments(boolean ignoreComments);
  public void    setCoalescing(boolean coalescing);
  public boolean isNamespaceAware();
  public boolean isValidating();
  public boolean isIgnoringElementContentWhitespace();
  public boolean isExpandEntityReferences();
  public boolean isIgnoringComments();
  public boolean isCoalescing();
  public void    setAttribute(String name, Object value)
   throws IllegalArgumentException;
  public Object  getAttribute(String name)
   throws IllegalArgumentException;

}

Different vendors provide different implementations of this abstract class. Java chooses the one to use based on the following conditions in order of preference:

  1. The value of the javax.xml.parsers.DocumentBuilderFactory Java system property

  2. The value of the javax.xml.parsers.DocumentBuilderFactory property specified in the lib/jaxp.properties properties file in the JRE directory

  3. The first value found in a META-INF/services/javax.xml.parsers.DocumentBuilderFactory file in the JAR files available to the runtime

  4. The platform default (org.apache.crimson.jaxp.DocumentBuilderFactoryImpl in Sun’s JDK 1.4).

SAXParser

SAXParser is an obsolete class for locating SAX 1 parsers and parsing documents. It’s been replaced by the org.xml.sax.helpers.XMLReaderFactory class in SAX2.

package javax.xml.parsers;

public abstract class SAXParser  {

  protected SAXParser();

  public void parse(InputStream in, HandlerBase handler)
   throws SAXException, IOException;
  public void parse(InputStream in, HandlerBase handler,
   String systemID) throws SAXException, IOException;
  public void parse(InputStream in, DefaultHandler handler)
   throws SAXException, IOException;
  public void parse(InputStream in, DefaultHandler handler,
   String systemID) throws SAXException, IOException;
  public void parse(String uri, HandlerBase handler)
   throws SAXException, IOException;
  public void parse(String uri, DefaultHandler handler)
   throws SAXException, IOException;
  public void parse(File f, HandlerBase handler)
   throws SAXException, IOException;
  public void parse(File f, DefaultHandler handler)
   throws SAXException, IOException;
  public void parse(InputSource in, HandlerBase handler)
   throws SAXException, IOException;
  public void parse(InputSource in, DefaultHandler handler)
   throws SAXException, IOException;

  public Parser    getParser() throws SAXException;
  public XMLReader getXMLReader() throws SAXException;
  public boolean   isNamespaceAware();
  public boolean   isValidating();
  public void      setProperty(String name, Object value)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public Object    getProperty(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException;

}

SAXParserFactory

SAXParserFactory is an obsolete class for building and configuring SAXParser objects in an implementation independent fashion. The concrete subclass to load is read from the javax.xml.parsers.SAXParserFactory Java system property. This class has been replaced by the org.xml.sax.helpers.XMLReaderFactory class in SAX2, and there's little reason to use it any more.

package javax.xml.parsers;

public abstract class SAXParserFactory  {

  protected SAXParserFactory();

  public static SAXParserFactory newInstance()
   throws FactoryConfigurationError;

  public SAXParser newSAXParser()
   throws ParserConfigurationException, SAXException;
  public void      setNamespaceAware(boolean awareness);
  public void      setValidating(boolean validating);
  public boolean   isNamespaceAware();
  public boolean   isValidating();
  public void      setFeature(String name, boolean value)
   throws ParserConfigurationException,
          SAXNotRecognizedException, SAXNotSupportedException;
  public boolean   getFeature(String name)
   throws ParserConfigurationException,
          SAXNotRecognizedException, SAXNotSupportedException;

}

Exceptions and Errors

The javax.xml.parsers package includes one error and one exception, representing the things that can go wrong when loading a parser or a DOM implementation. Either one is normally a symptom of class path problems.

ParserConfigurationException

A ParserConfigurationException signals that a factory is unable to load and instantiate a parser class.

package javax.xml.parsers;

public class ParserConfigurationException extends Exception {

  public ParserConfigurationException();
  public ParserConfigurationException(String message);

}
FactoryConfigurationError

A FactoryConfigurationError signals that Java is unable to load and instantiate the concrete factory class. This is normally a symptom of class path problems.

package javax.xml.parsers;

public class FactoryConfigurationError extends Error {

  public FactoryConfigurationError();
  public FactoryConfigurationError(String message);
  public FactoryConfigurationError(Exception e);
  public FactoryConfigurationError(Exception e, String message);

  public String    getMessage();
  public Exception getException();

}
Prev | Up | Next
DOM | Home | TrAX
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 03, 2002


TrAX
Prev Appendix A. XML APIs Quick Reference Next

TrAX

The Transformations API for XML (TrAX) integrates XSLT processing into Java in a manner that is independent both of the specific XSLT processor and the document model. It works equally well with DOM, JDOM, SAX, and raw streams. Indeed identity transforms can be used to transform one into the other.

javax.xml.transform

The javax.xml.transform package includes the base interfaces and abstract classes that provide abstract representations of both source and result XML documents and of the transformation itself. The subpackages specialize the sources and results to particular document models and APIs.

ErrorListener

ErrorListener is a callback interface Transformers use to report problems that occur during a transformation. There are three levels of errors: fatal errors that prevent the transformation from continuing, recoverable errors, and warnings. If a Transformer does not have an ErrorListener, then it prints error messages on System.err.

package javax.xml.transform;

public interface ErrorListener  {

  public void warning(TransformerException exception)
   throws TransformerException;
  public void error(TransformerException exception)
   throws TransformerException;
  public void fatalError(TransformerException exception)
   throws TransformerException;

}

OutputKeys

The OutputKeys class defines named constants for serialization parameters that are normally set by an xsl:output element in an XSLT stylesheet. In TrAX, these are used as the name arguments to the setOutputProperty() and getOutputProperty() methods of the Transformer class.

package javax.xml.transform;

public class OutputKeys  {

  public static final String METHOD = "method";
  public static final String VERSION = "version";
  public static final String ENCODING = "encoding";
  public static final String OMIT_XML_DECLARATION
   = "omit-xml-declaration";
  public static final String STANDALONE = "standalone";
  public static final String DOCTYPE_PUBLIC = "doctype-public";
  public static final String DOCTYPE_SYSTEM = "doctype-system";
  public static final String CDATA_SECTION_ELEMENTS
   = "cdata-section-elements";
  public static final String INDENT = "indent";
  public static final String MEDIA_TYPE = "media-type";

}

Result

The Result interface is a generic container for an XML document that will be produced by a transformation. Concrete classes implement this interface for SAX event sequences, DOM nodes, streams, and more.

package javax.xml.transform;

public interface Result  {

  public static final String PI_DISABLE_OUTPUT_ESCAPING;
  public static final String PI_ENABLE_OUTPUT_ESCAPING;

  public void   setSystemId(String url);
  public String getSystemId();

}

Source

The Source interface is a generic container for existing XML documents that will be used in a transformation as either the input document or the stylesheet. Concrete classes implement this interface for SAX event sequences, DOM nodes, streams, and more.

package javax.xml.transform;

public interface Source  {

  public void   setSystemId(String systemID);
  public String getSystemId();

}

SourceLocator

SourceLocator objects are used by the various kinds of TransformerException to indicate where in which file the problem that caused the exception lies.

package javax.xml.transform;

public interface SourceLocator  {

  public String getPublicId();
  public String getSystemId();
  public int    getLineNumber();
  public int    getColumnNumber();

}

Templates

Templates is a thread-safe class that represents a compiled stylesheet. It can quickly create new Transformer objects without having to reread and reparse the original stylesheet. It’s particularly useful when you want to use the same stylesheet in multiple threads.

package javax.xml.transform;

public interface Templates  {

  public Transformer newTransformer()
   throws TransformerConfigurationException;

  public Properties  getOutputProperties();

}

Transformer

Transformer is the class that represents a compiled stylesheet. It transforms Source objects into Result objects. A single Transformer can transform multiple input documents in sequence but not in parallel.

package javax.xml.transform;

public abstract class Transformer  {

  protected Transformer();

  public void transform(Source input, Result output)
   throws TransformerException;

  public void          setParameter(String name, Object value);
  public Object        getParameter(String name);
  public void          clearParameters();
  public void          setURIResolver(URIResolver resolver);
  public URIResolver   getURIResolver();
  public void          setOutputProperties(Properties serialization)
   throws IllegalArgumentException;
  public Properties    getOutputProperties();
  public void          setOutputProperty(String name,
   String value) throws IllegalArgumentException;
  public String        getOutputProperty(String name)
   throws IllegalArgumentException;
  public void          setErrorListener(ErrorListener listener)
   throws IllegalArgumentException;
  public ErrorListener getErrorListener();

}

TransformerFactory

TransformerFactory is an abstract factory that creates new Transformer and Templates objects. The concrete subclass that newInstance()instantiates is specified by the javax.xml.transform.TransformerFactory Java system property. If this class is not set, a platform dependent default class is chosen.

package javax.xml.transform;

public abstract class TransformerFactory  {

  protected TransformerFactory();

  public static TransformerFactory newInstance()
   throws TransformerFactoryConfigurationError;

  public Transformer newTransformer(Source source)
   throws TransformerConfigurationException;
  public Transformer newTransformer()
   throws TransformerConfigurationException;
  public Templates   newTemplates(Source source)
   throws TransformerConfigurationException;

  public Source        getAssociatedStylesheet(Source source,
   String media, String title, String charset)
   throws TransformerConfigurationException;
  public void          setURIResolver(URIResolver resolver);
  public URIResolver   getURIResolver();
  public boolean       getFeature(String name);
  public void          setAttribute(String name, Object value)
   throws IllegalArgumentException;
  public Object        getAttribute(String name)
   throws IllegalArgumentException;
  public void          setErrorListener(ErrorListener listener)
   throws IllegalArgumentException;
  public ErrorListener getErrorListener();

}

URIResolver

The XSLT processor passes any URLs encountered in the stylesheet’s xsl:import or xsl:include elements or referenced by the document() function to its URIResolver to give the program a chance to substitute a different resource. Returning null indicates that the default resolution mechanism should be used. The pattern is very similar to that of the SAX EntityResolver class. You can specify a URIResolver object by passing it to the setURIResolver() method of either Transformer or TransformerFactory.

package javax.xml.transform;

public interface URIResolver  {

  public Source resolve(String href, String base)
   throws TransformerException;

}

Exceptions and Errors

TrAX includes several exceptions and errors for signaling problems that occur during a transformation. Most TrAX methods wrap all checked exceptions that occur during processing in one of the TrAX exception classes. For instance, the transform() method does not throw an IOException if the network connection goes down while it’s reading a remote input document. Instead it throws a TransformerException.

TransformerConfigurationException

A TransformerConfigurationException is a checked exception thrown when an attempt to create a new Transformer or Templates object fails. The normal reason is a syntax error in the stylesheet or an IOException that prevents the stylesheet from being completely read.

package javax.xml.transform;

public class TransformerConfigurationException extends TransformerException {

  public TransformerConfigurationException();
  public TransformerConfigurationException(String message);
  public TransformerConfigurationException(Throwable t);
  public TransformerConfigurationException(String message,
   Throwable t);
  public TransformerConfigurationException(String message,
   SourceLocator locator);
  public TransformerConfigurationException(String message,
   SourceLocator locator, Throwable t);

}
TransformerException

TransformerException is a checked exception that signals a problem occurred during an attempted transformation. Possible causes include malformed input documents, and IOExceptions that prevent either the input document from being read or the output document from being written.

package javax.xml.transform;

public class TransformerException extends Exception {

  public TransformerException(String message);
  public TransformerException(Throwable t);
  public TransformerException(String message, Throwable t);
  public TransformerException(String message,
   SourceLocator locator);
  public TransformerException(String message,
   SourceLocator locator, Throwable t);

  public SourceLocator getLocator();
  public void          setLocator(SourceLocator location);
  public Throwable     getException();
  public Throwable     getCause();
  public Throwable     initCause(Throwable cause);
  public String        getMessageAndLocation();
  public String        getLocationAsString();
  public void          printStackTrace();
  public void          printStackTrace(PrintStream s);
  public void          printStackTrace(PrintWriter s);

}
TransformerFactoryConfigurationError

A TransformerFactoryConfigurationError indicates a problem with the TransformerFactory. The most common cause is that the specific concrete subclass of TransformerFactory indicated by the javax.xml.transform.TransformerFactory system property could not be found in the local class path. Unlike most errors, it’s not a bad idea to catch this and handle it appropriately.

package javax.xml.transform;

public class TransformerFactoryConfigurationError extends Error {

  public TransformerFactoryConfigurationError();
  public TransformerFactoryConfigurationError(String message);
  public TransformerFactoryConfigurationError(Exception e);
  public TransformerFactoryConfigurationError(Exception e,
   String message);

  public String    getMessage();
  public Exception getException();

}

javax.xml.transform.stream

The javax.xml.transform.stream package contains classes that wrap streams as either input to or output from a transformation. Supported streams include files, InputStreams and OutputStreams, Readers and Writers, files, and URLs.

StreamResult

A StreamResult directs the output of an XSLT transformation onto an OutputStream, Writer, URL, file, or other sink of bytes or characters.

package javax.xml.transform.stream;

public class StreamResult implements Result {

  public static final String FEATURE;

  public StreamResult();
  public StreamResult(OutputStream out);
  public StreamResult(Writer writer);
  public StreamResult(String systemID);
  public StreamResult(File f);

  public void         setOutputStream(OutputStream out);
  public OutputStream getOutputStream();
  public void         setWriter(Writer out);
  public Writer       getWriter();
  public void         setSystemId(String url);
  public void         setSystemId(File f);
  public String       getSystemId();

}

If possible, you should use an OutputStream or a File instead of a Writer so that TrAX can determine where it might need to emit a character reference instead of the actual character. If you specify more than one of these three possibilities, which one the processor writes to is implementation dependent.

StreamSource

A StreamSource provides input to an XSLT processor from an InputStream, Reader, URL, file, or other source of bytes or characters.

package javax.xml.transform.stream;

public class StreamSource implements Source {

  public static final String FEATURE;

  public StreamSource();
  public StreamSource(InputStream in);
  public StreamSource(InputStream in, String systemID);
  public StreamSource(Reader in);
  public StreamSource(Reader in, String systemID);
  public StreamSource(String url);
  public StreamSource(File f);

  public void        setInputStream(InputStream in);
  public InputStream getInputStream();
  public void        set Reader(Reader in);
  public Reader      get Reader();
  public void        setPublicId(String publicID);
  public String      getPublicId();
  public void        setSystemId(String url);
  public String      getSystemId();
  public void        setSystemId(File f);

}

You should always specify a system ID URL when creating a StreamSource. This is needed for resolving relative URLs that occur in the document. If you also specify an InputStream, File, or Reader then that actual content will be read from that source instead of the URL. If you specify more than one of these three possibilities, which one the processor reads from is implementation dependent.

javax.xml.transform.dom

The javax.xml.transform.dom package contains the classes needed to hook up DOM Node objects to TrAX transformations.

DOMLocator

The DOMLocator interface allows a Transformer reading from a DOMSource to report the actual DOM Node where an error occurred. To use this, you’ll need to cast the SourceLocator returned by the getLocator() method in Transformer to DOMLocator. (Naturally this will fail if the original source was not a DOMSource.)

package javax.xml.transform.dom;

public interface DOMLocator extends SourceLocator {

  public Node getOriginatingNode();

}

DOMResult

A DOMResult uses the output of an XSLT transformation to create a new DOM Document or DocumentFragment object.

package javax.xml.transform.dom;

public class DOMResult implements Result {

  public static final String FEATURE;

  public DOMResult();
  public DOMResult(Node node);
  public DOMResult(Node node, String systemID);

  public void   setNode(Node node);
  public Node   getNode();
  public void   setSystemId(String url);
  public String getSystemId();

}

DOMSource

A DOMSource provides input to an XSLT processor from a DOM Node object. In practice, only DOM Document objects can be reliably transformed, not arbitrary types of nodes.

package javax.xml.transform.dom;

public class DOMSource implements Source {

  public static final String FEATURE;

  public DOMSource();
  public DOMSource(Node node);
  public DOMSource(Node node, String systemID);

  public void   setNode(Node node);
  public Node   getNode();
  public void   setSystemId(String url);
  public String getSystemId();

}

javax.xml.transform.sax

The javax.xml.transform.sax package contains the classes needed to hook up SAX event sequences (normally represented as ContentHandlers) to TrAX transformations.

SAXResult

A SAXResult passes the output of an XSLT transformation into a SAX ContentHandler (and optionally a LexicalHandler). This allows you to post-process the output of a transform with a SAX program.

package javax.xml.transform.sax;

public class SAXResult implements Result {

  public static final String FEATURE;

  public SAXResult();
  public SAXResult(ContentHandler handler);

  public void           setHandler(ContentHandler handler);
  public ContentHandler getHandler();
  public void          setLexicalHandler(LexicalHandler handler);
  public LexicalHandler getLexicalHandler();
  public void           setSystemId(String systemID);
  public String         getSystemId();

}

SAXSource

A SAXSource provides input to an XSLT processor from a SAX event sequence. This is especially useful when you want to apply a SAX filter to a document before transforming it.

package javax.xml.transform.sax;

public class SAXSource implements Source {

  public static final String FEATURE;

  public SAXSource();
  public SAXSource(XMLReader reader, InputSource input);
  public SAXSource(InputSource input);

  public void        setXMLReader(XMLReader reader);
  public XMLReader   getXMLReader();
  public void        setInputSource(InputSource input);
  public InputSource getInputSource();
  public void        setSystemId(String url);
  public String      getSystemId();

  public static InputSource sourceToInputSource(Source source);

}

SAXTransformerFactory

The SAXTransformerFactory class allows you to create Transformer and Templates objects that apply a SAX filter to the source document before transforming it.

package javax.xml.transform.sax;

public abstract class SAXTransformerFactory
 extends TransformerFactory {

  public static final String FEATURE;
  public static final String FEATURE_XMLFILTER;

  protected SAXTransformerFactory();

  public TransformerHandler newTransformerHandler(Source source)
   throws TransformerConfigurationException;
  public TransformerHandler newTransformerHandler(
   Templates templates) throws TransformerConfigurationException;
  public TransformerHandler newTransformerHandler()
   throws TransformerConfigurationException;
  public TemplatesHandler   newTemplatesHandler()
   throws TransformerConfigurationException;
  public XMLFilter          newXMLFilter(Source source)
   throws TransformerConfigurationException;
  public XMLFilter          newXMLFilter(Templates templates)
   throws TransformerConfigurationException;

}

TemplatesHandler

TemplatesHandler is a ContentHandler that builds a Templates object from a SAX event sequence that reads a stylesheet. Client applications do not normally need to use this class directly.

package javax.xml.transform.sax;

public interface TemplatesHandler extends ContentHandler {

  public Templates getTemplates();
  public void      setSystemId(String url);
  public String    getSystemId();

}

TransformerHandler

TransformerHandler objects receive SAX events from a transformation and convert them into output. Client applications do not normally need to use this class directly.

package javax.xml.transform.sax;

public interface TransformerHandler
 extends ContentHandler, LexicalHandler, DTDHandler {

  public void        setResult(Result result)
   throws IllegalArgumentException;
  public void        setSystemId(String url);
  public String      getSystemId();
  public Transformer getTransformer();

}
Prev | Up | Next
JAXP | Home | JDOM Quick Reference
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 13, 2002


JDOM Quick Reference
Prev Appendix A. XML APIs Quick Reference Next

JDOM Quick Reference

JDOM is an open source, pure Java API for reading, writing, and manipulating XML documents. This section provides complete signatures for the documented, public parts of JDOM, with the single exception of the org.jdom.adapters package, which is really intended only for internal use in JDOM.

Caution

JDOM is not finished at the time of this writing. This appendix is based on an early development release of JDOM beta 9, current as of May 17, 2002. I expect some of the details here to change before JDOM is released. (In fact, I’m actively lobbying for a few of them to change.)

org.jdom

The org.jdom package contains the core classes that JDOM uses to model the XML tree, one class per node type, as well as some generically useful classes such as Verifier, JDOMFactory, and JDOMException.

Attribute

The Attribute class represents an attribute node. Each attribute has a local name, a namespace (which may be null), a string value, a type, and a parent Element. Attribute types are represented by the named int constants in this class.

package org.jdom;

public class Attribute implements Serializable, Cloneable {

  public static final int UNDECLARED_ATTRIBUTE;
  public static final int CDATA_ATTRIBUTE;
  public static final int ID_ATTRIBUTE;
  public static final int IDREF_ATTRIBUTE;
  public static final int IDREFS_ATTRIBUTE;
  public static final int ENTITY_ATTRIBUTE;
  public static final int ENTITIES_ATTRIBUTE;
  public static final int NMTOKEN_ATTRIBUTE;
  public static final int NMTOKENS_ATTRIBUTE;
  public static final int NOTATION_ATTRIBUTE;
  public static final int ENUMERATED_ATTRIBUTE;

  protected String    name;
  protected Namespace namespace;
  protected String    value;
  protected int       type;
  protected Object    parent;

  protected Attribute();
  public    Attribute(
   String name, String value, Namespace namespace)
   throws IllegalNameException, IllegalDataException;
  public    Attribute(
   String name, String value, int type, Namespace namespace)
   throws IllegalNameException, IllegalDataException;
  public    Attribute(String name, String value)
   throws IllegalNameException, IllegalDataException;
  public    Attribute(String name, String value, int type)
   throws IllegalNameException, IllegalDataException;

  public    Element   getParent();
  protected Attribute setParent(Element parent);
  public    Document  getDocument();
  public    Attribute detach();
  public    String    getName();
  public    Attribute setName(String name)
   throws IllegalNameException;
  public    String    getQualifiedName();
  public    String    getNamespacePrefix();
  public    String    getNamespaceURI();
  public    Namespace getNamespace();
  public    Attribute setNamespace(Namespace namespace)
   throws IllegalNameException;
  public    String    getValue();
  public    Attribute setValue(String value);
  public    int       getAttributeType();
  public    Attribute setAttributeType(int type)
   throws IllegalDataException;

  // Java utility methods
  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public       Object  clone();

  // Convenience methods for converting the value to various
  // primitive types
  public int     getIntValue() throws DataConversionException;
  public long    getLongValue() throws DataConversionException;
  public float   getFloatValue() throws DataConversionException;
  public double  getDoubleValue() throws DataConversionException;
  public boolean getBooleanValue()
   throws DataConversionException;

 }

CDATA

The CDATA class is a subclass of Text (from which it inherits most of its functionality) that represents a CDATA section. The only real difference between this class and Text is that an XMLOutputter will use a CDATA section to write out the contents of a CDATA object rather than escaping characters like < with entity and character references.

package org.jdom;

public class CDATA extends Text {

  protected CDATA();
  public    CDATA(String s);

  public Text setText(String s) throws IllegalDataException;
  public void append(String s) throws IllegalDataException;

  public String toString();

 }

Comment

The Comment class represents a comment node. Each Comment object contains the text of the comment, a parent Element (which will be null if this comment is in the prolog or epilog), and an owner Document.

package org.jdom;

public class Comment implements Serializable, Cloneable {

  protected String text;
  protected Object parent;

  protected Comment();
  public    Comment(String text) throws IllegalDataException;

  public    Element  getParent();
  protected Comment  setParent(Element parent);
  public    Comment  detach();
  public    Document getDocument();
  protected Comment  setDocument(Document document);
  public    String   getText();
  public    Comment  setText(String text)
   throws IllegalDataException;

  // Java utility methods
  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public       Object  clone();

 }

DocType

The DocType class represents a document type declaration. Each DocType object contains the declared root element name, the public ID and the system ID (both of which may be null), the owner document, and a String containing the internal DTD subset.

package org.jdom;

public class DocType implements Serializable, Cloneable {

  protected String   elementName;
  protected String   publicID;
  protected String   systemID;
  protected Document document;
  protected String   internalSubset;

  protected DocType();
  public    DocType(String elementName, String publicID,
   String systemID)
   throws IllegalNameException, IllegalDataException;
  public    DocType(String elementName, String systemID) throws IllegalNameException;
  public    DocType(String elementName)
   throws IllegalNameException;

  public    String   getElementName();
  public    DocType  setElementName(String elementName)
   throws IllegalNameException;
  public    String   getPublicID();
  public    DocType  setPublicID(String publicID)
   throws IllegalDataException;;
  public    String   getSystemID();
  public    DocType  setSystemID(String url);
  public    Document getDocument();
  protected DocType  setDocument(Document document);
  public    void     setInternalSubset(String declarations);
  public    String   getInternalSubset();

  // Java utility methods
  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public       Object  clone();

 }

Document

The Document class represents a complete document and serves as the root of the JDOM tree. Each Document object contains a list of its content and the document’s DocType (if it has one). Each Document should have exactly one Element in its content list. However, documents may be temporarily rootless. Almost anything you do to such a Document other than setting the root element will throw an IllegalStateException.

package org.jdom;

public class Document implements Serializable, Cloneable {

  protected ContentList content;
  protected DocType     docType;

  public Document();
  public Document(Element rootElement, DocType docType)
   throws IllegalAddException;
  public Document(Element rootElement);
  public Document(List newContent, DocType docType)
   throws IllegalAddException;
  public Document(List content) throws IllegalAddException;

  public boolean  hasRootElement();
  public Element  getRootElement() throws IllegalStateException;
  public Document setRootElement(Element rootElement);
  public Element  detachRootElement();
  public DocType  getDocType();
  public Document setDocType(DocType docType)
   throws IllegalAddException;
  public Document addContent(ProcessingInstruction pi);
  public Document addContent(Comment comment);
  public List     getContent();
  public List     getContent(Filter filter);
  public Document setContent(List newContent)
   throws IllegalAddException;
  public boolean  removeContent(ProcessingInstruction pi);
  public boolean  removeContent(Comment comment);

  // Java utility methods
  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public       Object  clone();

 }

Element

The Element class represents a complete element. Each element has a local name, a namespace (which may be null), a parent Element (which is null if this is the root element or not currently part of a document), a list of its children, a list of its attributes, and a list of any namespace prefixes declared on the element that are not used by the element or one of its attributes.

package org.jdom;

public class Element implements Serializable, Cloneable {

  protected String        name;
  protected Namespace     namespace;
  protected List          additionalNamespaces;
  protected Object        parent;
  protected AttributeList attributes;
  protected ContentList   content;

  protected Element();
  public    Element(String name, Namespace namespace)
   throws IllegalNameException;
  public    Element(String name) throws IllegalNameException;
  public    Element(String name, String uri)
   throws IllegalNameException;
  public    Element(String name, String prefix, String uri)
   throws IllegalNameException;

  public    String    getName();
  public    Element   setName(String name)
   throws IllegalNameException;
  public    Namespace getNamespace();
  public    Element   setNamespace(Namespace namespace);
  public    String    getNamespacePrefix();
  public    String    getNamespaceURI();
  public    Namespace getNamespace(String prefix);
  public    String    getQualifiedName();
  public    void      addNamespaceDeclaration(
   Namespace additional);
  public    void      removeNamespaceDeclaration(
   Namespace additionalNamespace);
  public    List      getAdditionalNamespaces();

  public    Element   getParent();
  protected Element   setParent(Element parent);
  public    Element   detach();
  public    boolean   isRootElement();
  protected Element   setDocument(Document document);
  public    Document  getDocument();
  public    boolean   isAncestor(Element element);

  public    String    getText();
  public    Element   setText(String text);
  public    String    getTextTrim();
  public    String    getTextNormalize();

  public    String    getChildText(String name);
  public    String    getChildTextTrim(String name);
  public    String    getChildTextNormalize(String name);
  public    String    getChildText(String name, Namespace ns);
  public    String    getChildTextTrim(String name,
   Namespace namespace);
  public    String    getChildTextNormalize(String name,
   Namespace namespace);

  public    List      getContent();
  public    List      getContent(Filter filter);
  public    Element   setContent(List newContent)
   throws IllegalAddException;
  public    List      getChildren();
  public    List      getChildren(String name);
  public    List      getChildren(String name, Namespace ns);
  public    Element   getChild(String name, Namespace ns);
  public    Element   getChild(String name);
  public    Element   addContent(String s)
   throws IllegalAddException;
  public    Element   addContent(Text text)
   throws IllegalAddException;
  public    Element   addContent(Element element)
   throws IllegalAddException;
  public    Element   addContent(ProcessingInstruction pi)
   throws IllegalAddException;
  public    Element   addContent(EntityRef ref)
   throws IllegalAddException;
  public    Element   addContent(Comment comment)
   throws IllegalAddException;
  public    boolean   removeChild(String name);
  public    boolean   removeChild(String name, Namespace ns);
  public    boolean   removeChildren(String name);
  public    boolean   removeChildren(String name, Namespace ns);
  public    boolean   removeContent(Element element);
  public    boolean   removeContent(ProcessingInstruction pi);
  public    boolean   removeContent(Comment comment);
  public    boolean   removeContent(Text text);
  public    boolean   removeContent(EntityRef entity);

  // Attribute methods
  public    List      getAttributes();
  public    Attribute getAttribute(String name);
  public    Attribute getAttribute(String name, Namespace ns);
  public    String    getAttributeValue(String name);
  public    String    getAttributeValue(String name,
   Namespace ns, String def);
  public    String    getAttributeValue(String name, String def);
  public    String    getAttributeValue(String name, Namespace ns);
  public    Element   setAttributes(List newAttributes)
   throws IllegalAddException;
  public    Element   setAttribute(String name, String value)
   throws IllegalNameException, IllegalDataException;
  public    Element   setAttribute(String name, String value, Namespace ns)
   throws IllegalNameException, IllegalDataException;
  public    Element   setAttribute(Attribute attribute);
  public    boolean   removeAttribute(String name);
  public    boolean   removeAttribute(String name, Namespace ns);
  public    boolean   removeAttribute(Attribute attribute);

  // Java utility methods
  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public       Object  clone();

 }

EntityRef

The EntityRef class represents an unexpanded entity reference, such as might be produced by a non-validating parser which does not read the external DTD subset. Entity references for which the replacement text is known are not included as EntityRef objects. Instead their replacement text is parsed and included.

package org.jdom;

public class EntityRef implements Serializable, Cloneable {

  protected String name;
  protected String publicID;
  protected String systemID;
  protected Object parent;

  protected EntityRef();
  public    EntityRef(String name);
  public    EntityRef(String name, String systemID);
  public    EntityRef(String name, String publicID,
   String systemID);

  public    String    getName();
  public    EntityRef setName(String name)
   throws IllegalNameException;
  public    String    getPublicID();
  public    String    getSystemID();
  public    EntityRef setPublicID(String newPublicID)
   throws IllegalDataException;
  public    EntityRef setSystemID(String url)
   throws IllegalDataException;
  public    Document  getDocument();
  public    Element   getParent();
  public    EntityRef detach();
  protected EntityRef setParent(Element parent);

  // Java utility methods
  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public       Object  clone();

 }

Namespace

The Namespace class encapsulates a namespace URI and possibly a prefix. It uses the flyweight design pattern so that twenty different elements in the same namespace share only one Namespace object between them.

package org.jdom;

public final class Namespace  {

  // empty string
  public static final Namespace NO_NAMESPACE;

  // http://www.w3.org/XML/1998/namespace
  public static final Namespace XML_NAMESPACE;

  // The constructor is private

  // Factory methods
  public static Namespace getNamespace(String prefix, String uri)
   throws IllegalNameException;
  public static Namespace getNamespace(String uri)
   throws IllegalNameException;

  public String getPrefix();
  public String getURI();

  // Java utility methods
  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();

 }

ProcessingInstruction

The ProcessingInstruction class represents a complete processing instruction. Each ProcessingInstruction has a target, data, a parent Element (which will be null if this instruction is in the prolog or epilog), and an owner Document. If the data is stored in pseudo-attributes, then there’s also a map containing the pseudo-attributes separated into names and values. However, not all processing instructions can be represented in this format.

package org.jdom;

public class ProcessingInstruction
 implements Serializable, Cloneable {

  protected String target;
  protected String rawData;
  protected Map    mapData;
  protected Object parent;

  protected ProcessingInstruction();
  public    ProcessingInstruction(String target, Map data)
   throws IllegalTargetException;
  public    ProcessingInstruction(String target, String data)
   throws IllegalTargetException;

  public    ProcessingInstruction setTarget(String newTarget)
   throws IllegalTargetException;
  public    String                getTarget();
  public    Element               getParent();
  protected ProcessingInstruction setParent(Element parent);
  public    ProcessingInstruction detach();
  public    Document              getDocument();
  protected ProcessingInstruction setDocument(Document document);
  public    String                getData();
  public    ProcessingInstruction setData(String data);

  public    ProcessingInstruction setData(Map data);
  public    List                  getNames();
  public    String                getValue(String name);
  public    ProcessingInstruction setValue(String name,
   String value);
  public    boolean               removeValue(String name);

  // Java utility methods
  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public       Object  clone();

 }

Text

The Text class represents a text node. In general, Text objects are not guaranteed to contain the maximum possible contiguous run of text, though this will be true of documents that have been created by a SAXBuilder and not modified since. In most normal usage you can ignore this class and use strings instead.

package org.jdom;

public class Text implements Serializable, Cloneable {

  protected String value;
  protected Object parent;

  protected Text();
  public    Text(String s) throws IllegalDataException;

  public String   getText();
  public String   getTextTrim();
  public String   getTextNormalize();
  public Text     setText(String s) throws IllegalDataException;
  public void     append(String s) throws IllegalDataException;
  public void     append(Text text) throws IllegalDataException;

  public Element  getParent();
  public Document getDocument();
  protected Text  setParent(Element parent);
  public Text     detach();

  public static String normalizeString(String s);

  // Java utility methods
  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public       Object  clone();

 }

Verifier

Verifier is a utility class the other JDOM classes rely on to decide whether or not particular strings are acceptable for particular purposes. For example, TimeLimit is a legal element name but Time Limit is not. You don’t normally need to use this class directly. If you do, you should note that the various check methods all return null if the argument passes the test and a non-empty string containing an error message if the test fails.

package org.jdom;

public final class Verifier  {

  public static String checkElementName(String name);
  public static String checkAttributeName(String name);
  public static String checkCharacterData(String text);
  public static String checkCDATASection(String data);
  public static String checkNamespacePrefix(String prefix);
  public static String checkNamespaceURI(String uri);
  public static String checkNamespaceCollision(
   Namespace namespace, Namespace other);
  public static String checkNamespaceCollision(
   Attribute attribute, Element element);
  public static String checkNamespaceCollision(
   Namespace namespace, Element element);
  public static String checkNamespaceCollision(
   Namespace namespace, Attribute attribute);
  public static String checkNamespaceCollision(
   Namespace namespace, List list);
  public static String checkProcessingInstructionTarget(
   String target);
  public static String checkCommentData(String data);
  public static String checkPublicID(String publicID);
  public static String checkSystemLiteral(String systemID);
  public static String checkXMLName(String name);

  public static boolean isXMLCharacter(char c);
  public static boolean isXMLNameCharacter(char c);
  public static boolean isXMLNameStartCharacter(char c);
  public static boolean isXMLLetterOrDigit(char c);
  public static boolean isXMLLetter(char c);
  public static boolean isXMLCombiningChar(char c);
  public static boolean isXMLExtender(char c);
  public static boolean isXMLDigit(char c);

 }

Exceptions and Errors

JDOM defines a number of unique exceptions. JDOMException is the basic superclass for most checked exceptions JDOM methods can throw. A few methods in the input and output packages also throw standard IOExceptions. Finally, JDOM provides several subclasses of IllegalArgumentException (a runtime exception) that are thrown when a program attempts to set XML constructs like element names to illegal values.

DataConversionException

DataConversionException is thrown by the five getInt/Long/Float/Double/BooleanValue() methods in Attribute when the attribute value cannot be parsed as the requested type. This is a checked exception.

package org.jdom;

public class DataConversionException extends JDOMException {

  public DataConversionException(String message, String type);

 }
IllegalAddException

IllegalAddException is thrown when code attempts to add a node where it doesn’t belong. This could be because the new child already has a parent or because the new child can never be placed where you’re trying to fit it (e.g. adding Text object as a child of a Document). This is a runtime exception.

package org.jdom;

public class IllegalAddException
 extends IllegalArgumentException {

  public IllegalAddException(Element base, Attribute added,
   String reason);
  public IllegalAddException(Element base, Element added,
   String reason);
  public IllegalAddException(Document base, Element added,
   String reason);
  public IllegalAddException(Element base,
   ProcessingInstruction added, String reason);
  public IllegalAddException(Document base,
   ProcessingInstruction added, String reason);
  public IllegalAddException(Element base, Comment added,
   String reason);
  public IllegalAddException(Element base, CDATA added,
   String reason);
  public IllegalAddException(Element base, Text added,
   String reason);
  public IllegalAddException(Document base, Comment added,
   String reason);
  public IllegalAddException(Element base, EntityRef added,
   String reason);
  public IllegalAddException(Element base, Namespace added,
   String reason);
  public IllegalAddException(Document base, DocType added,
   String reason);
  public IllegalAddException(String reason);

 }
IllegalDataException

An IllegalDataException is thrown when code attempts to set some text content to a string that does not satisfy XML’s well-formedness rules. Exactly what these rules are depends on context. For instance, a comment cannot contain the two hyphen string --. No content can include the ASCII vertical tab or bell characters, and so forth. The data argument contains the illegal text. This is a runtime exception.

package org.jdom;

public class IllegalDataException
 extends IllegalArgumentException {

  public IllegalDataException(String data, String construct,
   String reason);
  public IllegalDataException(String data, String construct);

 }
IllegalNameException

An IllegalNameException is thrown when code attempts to set the name of an attribute, element, or entity reference to a string that is not a namespace well-formed XML name. This is a runtime exception.

package org.jdom;

public class IllegalNameException
 extends IllegalArgumentException {

  public IllegalNameException(String name, String construct,
   String reason);
  public IllegalNameException(String name, String construct);

 }
IllegalTargetException

An IllegalTargetException is thrown when code attempts to set the target of a processing instruction to a string that is not a legal XML name. This is a runtime exception.

package org.jdom;

public class IllegalTargetException
 extends IllegalArgumentException {

  public IllegalTargetException(String target,
   String reason);
  public IllegalTargetException(String target);

 }
JDOMException

JDOMException is the common superclass for the different kinds of checked exceptions that may be thrown while working with JDOM.

package org.jdom;

public class JDOMException extends Exception {

  protected Throwable cause;

  public JDOMException();
  public JDOMException(String message);
  public JDOMException(String message, Throwable cause);

  public Throwable initCause(Throwable cause);
  public Throwable getCause();
  public String    getMessage();
  public void      printStackTrace();
  public void      printStackTrace(PrintStream out);
  public void      printStackTrace(PrintWriter out);

 }

org.jdom.filter

JDOM uses the filter package internally to make sure client code doesn’t do silly things like adding a java.io.InputStream to an Element’s children, or slightly less silly but still illegal things such as adding a Text object to a Document’s children. You can also use filters in your own code to simplify navigation and search in XML documents.

ContentFilter

ContentFilter is the basic filter that allows you to specify what kinds of nodes you want to pass the filter. The actual filter is stored as an int mask. The individual node types are integral powers of two that set exactly one bit in the mask. As usual with bit masks, you can combine the different fields with the bitwise or operator |.

package org.jdom.filter;

public class ContentFilter implements Filter {

  public static final int ELEMENT;
  public static final int CDATA;
  public static final int TEXT;
  public static final int COMMENT;
  public static final int PI;
  public static final int ENTITYREF;
  public static final int DOCUMENT;

  protected int filterMask;

  public ContentFilter();
  public ContentFilter(boolean allVisible);
  public ContentFilter(int mask);

  public int  getFilterMask();
  public void setFilterMask(int mask);
  public void setDefaultMask();

  public void setDocumentContent();
  public void setElementContent();
  public void setElementVisible(boolean visible);
  public void setCDATAVisible(boolean visible);
  public void setTextVisible(boolean visible);
  public void setCommentVisible(boolean visible);
  public void setPIVisible(boolean visible);
  public void setEntityRefVisible(boolean visible);

  public boolean canAdd(Object o);
  public boolean canRemove(Object o);
  public boolean matches(Object o);
  public boolean equals(Object o);

 }

ElementFilter

The ElementFilter class enables you to define filters that pass only elements with a certain name, or a certain namespace, or a certain name in a certain namespace.

Note

It is likely that the behavior of the ElementFilter(String name) constructor will be changed so that it creates a filter that selects elements with that name and no namespace rather than elements with that name in any namespace.

package org.jdom.filter;

public class ElementFilter implements Filter {

  protected String name;
  protected Namespace namespace;

  public ElementFilter();
  public ElementFilter(String name);
  public ElementFilter(Namespace namespace);
  public ElementFilter(String name, Namespace namespace);

  public boolean canAdd(Object o);
  public boolean canRemove(Object o);
  public boolean matches(Object o);
  public boolean equals(Object o);

 }

Filter

The Filter interface identifies which child nodes should appear in a JDOM list. Node objects for which matches() returns false are filtered out of the list.

Caution

The canAdd() and canRemove() methods are probably going to be deprecated and then removed in the near future.

package org.jdom.filter;

public interface Filter  {

  public boolean canAdd(Object o);
  public boolean canRemove(Object o);
  public boolean matches(Object o);

 }

org.jdom.input

The org.jdom.input class contains classes involved with building JDOM objects from other formats such as XML files, DOM Document objects, and such.

BuilderErrorHandler

The BuilderErrorHandler class is used to report errors that occur during SAX parsing. By default it rethrows the exception to halt parsing for errors and fatal errors and ignores warnings. You don’t normally need to interact with this class directly.

package org.jdom.input;

public class BuilderErrorHandler implements org.xml.sax.ErrorHandler {

  public BuilderErrorHandler();

  public void warning(SAXParseException exception)
   throws SAXException;
  public void error(SAXParseException exception)
   throws SAXException;
  public void fatalError(SAXParseException exception)
   throws SAXException;

 }

DefaultJDOMFactory

The DefaultJDOMFactory class is used by SAXBuilder and DOMBuilder to create standard, undecorated JDOM trees. You rarely need to use this class directly.

package org.jdom.input;

public class DefaultJDOMFactory implements JDOMFactory {

  public DefaultJDOMFactory();

  public Attribute             attribute(String name,
   String value, Namespace namespace)
   throws IllegalNameException, IllegalDataException;
  public Attribute             attribute(String name,
   String value, int type, Namespace namespace)
   throws IllegalNameException, IllegalDataException;
  public Attribute             attribute(String name,
   String value) throws IllegalNameException, IllegalDataException;
  public Attribute             attribute(String name,
   String value, int type)
   throws IllegalNameException, IllegalDataException;
  public CDATA                 cdata(String text)
   throws IllegalDataException;
  public Text                  text(String text)
   throws IllegalDataException;
  public Comment               comment(String text)
   throws IllegalDataException;
  public DocType               docType(String elementName,
   String publicID, String systemID)
   throws IllegalNameException, IllegalDataException;
  public DocType               docType(String elementName,
   String systemID) throws IllegalNameException, IllegalDataException;
  public DocType               docType(String elementName)
   throws IllegalNameException;
  public Document              document(Element rootElement,
   DocType docType) IllegalAddException;
  public Document              document(Element rootElement)
   throws IllegalAddException;
  public Element               element(String name,
   Namespace namespace) throws IllegalNameException;
  public Element               element(String name)
   throws IllegalNameException;
  public Element               element(String name, String uri)
   throws IllegalNameException;
  public Element               element(String name,
   String prefix, String uri) throws IllegalNameException;
  public ProcessingInstruction processingInstruction(
   String target, Map data) throws IllegalTargetException;
  public ProcessingInstruction processingInstruction(
   String target, String data) throws IllegalTargetException;
  public EntityRef             entityRef(String name)
   throws IllegalNameException;
  public EntityRef             entityRef(String name,
   String publicID, String systemID)
   throws IllegalNameException, IllegalTargetException;

 }

DOMBuilder

The DOMBuilder class converts DOM org.w3c.dom.Document objects into org.jdom.Document objects and DOM org.w3c.dom.Element objects into org.jdom.Element objects. It’s useful for interoperating with DOM programs and libraries.

package org.jdom.input;

public class DOMBuilder  {

  public DOMBuilder();
  public DOMBuilder(String adapterClass);

  public void setFactory(JDOMFactory factory);

  public Document build(org.w3c.dom.Document domDocument)
   throws IllegalDataException;
  public Element  build(org.w3c.dom.Element  domElement)
   throws IllegalDataException;

 }

JDOMFactory

DOMBuilder and SAXBuilder rely on a JDOMFactory to build node objects. You can implement this class in order to have the builder objects build subclasses of your own devising rather than the standard JDOM classes. This interface is for advanced use, and then only when subclassing the standard org.jdom classes.

package org.jdom.input;

public interface JDOMFactory  {

  public Attribute             attribute(String name,
   String value, Namespace namespace);
  public Attribute             attribute(String name,
   String value, int type, Namespace namespace);
  public Attribute             attribute(String name,
   String value);
  public Attribute             attribute(String name,
   String value, int type);
  public CDATA                 cdata(String text);
  public Text                  text(String text);
  public Comment               comment(String text);
  public DocType               docType(String elementName,
   String publicID, String systemID);
  public DocType               docType(String elementName,
   String systemID);
  public DocType               docType(String elementName);
  public Document              document(Element rootElement,
   DocType docType);
  public Document              document(Element rootElement);
  public Element               element(String name,
   Namespace namespace);
  public Element               element(String name);
  public Element               element(String name, String uri);
  public Element               element(String name,
   String prefix, String uri);
  public ProcessingInstruction processingInstruction(
   String target, Map data);
  public ProcessingInstruction processingInstruction(
   String target, String data);
  public EntityRef             entityRef(String name);
  public EntityRef             entityRef(String name,
   String publicID, String systemID);

 }

SAXBuilder

SAXBuilder is the preferred means of parsing XML documents into JDOM. It relies on an underlying SAX parser, but is agnostic about which parser is used.

package org.jdom.input;

public class SAXBuilder  {

  protected JDOMFactory factory;

  public SAXBuilder();
  public SAXBuilder(boolean validate);
  public SAXBuilder(String saxDriverClass);
  public SAXBuilder(String saxDriverClass, boolean validate);

  public void setFactory(JDOMFactory factory);
  public void setValidation(boolean validate);
  public void setErrorHandler(ErrorHandler errorHandler);
  public void setEntityResolver(EntityResolver entityResolver);
  public void setDTDHandler(DTDHandler dtdHandler);
  public void setXMLFilter(XMLFilter xmlFilter);
  public void setIgnoringElementContentWhitespace(
   boolean ignoringWhite);
  public void setFeature(String name, boolean value);
  public void setProperty(String name, Object value);
  public void setExpandEntities(boolean expand);

  public Document build(InputSource in)
   throws JDOMException, IOException;
  public Document build(InputStream in)
   throws JDOMException, IOException;
  public Document build(File file)
   throws JDOMException, IOException;
  public Document build(URL url)
   throws JDOMException, IOException;
  public Document build(InputStream in, String systemID)
   throws JDOMException, IOException;
  public Document build(Reader characterStream)
   throws JDOMException, IOException;
  public Document build(Reader characterStream, String SystemID)
   throws JDOMException, IOException;
  public Document build(String systemID)
   throws JDOMException, IOException;

  protected SAXHandler createContentHandler();
  protected void       configureContentHandler(
   SAXHandler contentHandler);
  protected XMLReader  createParser() throws JDOMException;
  protected void       configureParser(XMLReader parser,
   SAXHandler contentHandler) throws JDOMException;
  protected URL        fileToURL(File f)
   throws MalformedURLException;

 }

SAXHandler

SAXBuilder uses SAXHandler to receive information from the underlying SAX parser. It’s extremely rare that you need to use this class yourself.

package org.jdom.input;

public class SAXHandler extends DefaultHandler
 implements LexicalHandler, DeclHandler, DTDHandler {

  protected Stack      stack;
  protected boolean    atRoot;
  protected boolean    inDTD;
  protected boolean    inInternalSubset;
  protected boolean    previousCDATA;
  protected boolean    inCDATA;
  protected boolean    suppress;
  protected LinkedList declaredNamespaces;
  protected LinkedList availableNamespaces;

  public SAXHandler();
  public SAXHandler(JDOMFactory factory);

  public Document    getDocument();
  public JDOMFactory getFactory();
  public Locator     getDocumentLocator();

  public void    setExpandEntities(boolean expand);
  public boolean getExpandEntities();
  public void    setIgnoringElementContentWhitespace(
   boolean ignoringWhite);
  public boolean getIgnoringElementContentWhitespace();

  // DeclHandler methods
  public void    externalEntityDecl(String name,
   String publicID, String systemID)
   throws SAXException;
  public void    attributeDecl(String elementName,
   String attributeName, String type, String valueDefault, String value)
   throws SAXException;
  public void    elementDecl(String name, String model) throws SAXException;
  public void    internalEntityDecl(String name, String value) throws SAXException;

  // ContentHandler methods
  public void setDocumentLocator(Locator locator);
  public void startPrefixMapping(String prefix, String uri)
   throws SAXException;
  public void endPrefixMapping(String prefix) throws SAXException;
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts)
    throws SAXException;
  public void characters(char ch, int start, int length)
   throws SAXException;
  public void ignorableWhitespace(char ch, int start, int length)
   throws SAXException;
  public void skippedEntity(String name) throws SAXException;
  public void processingInstruction(String target, String data)
   throws SAXException;
  public void endElement(String namespaceURI, String localName,
   String qualifiedName) throws SAXException;

  // LexicalHandler methods
  public void startDTD(String name, String publicID, String systemID)
   throws SAXException;
  public void endDTD() throws SAXException;
  public void startEntity(String name) throws SAXException;
  public void endEntity(String name) throws SAXException;
  public void startCDATA() throws SAXException;
  public void endCDATA() throws SAXException;
  public void comment(char ch, int start, int length)
   throws SAXException;

  // DTDHandler methods
  public void notationDecl(String name, String publicID,
   String systemID) throws SAXException;
  public void unparsedEntityDecl(String name, String publicID,
   String systemID, String notationName) throws SAXException;

  // Internal methods
  protected void    flushCharacters() throws SAXException;
  protected void    appendExternalId(String publicID,
   String systemID);
  protected Element getCurrentElement() throws SAXException;

 }

org.jdom.output

The org.jdom.output package is responsible for converting JDOM Document objects into other forms such as files, streams of text, DOM Document objects, and SAX event sequences.

DOMOutputter

The DOMOutputter class converts a JDOM Document object to a DOM Document object.

package org.jdom.output;

public class DOMOutputter  {

  public DOMOutputter();
  public DOMOutputter(String adapterClass);

  public org.w3c.dom.Document output(Document document)
   throws JDOMException;

  protected org.w3c.dom.Element output(Element element,
   org.w3c.dom.Document factory, NamespaceStack namespaces)
   throws JDOMException;
  protected org.w3c.dom.Attr    output(Attribute attribute,
   org.w3c.dom.Document factory) throws JDOMException;

 }

SAXOutputter

The SAXOutputter class walks a JDOM Document tree while firing events at a SAX ContentHandler.

package org.jdom.output;

public class SAXOutputter  {

  public SAXOutputter();
  public SAXOutputter(ContentHandler contentHandler);
  public SAXOutputter(ContentHandler contentHandler,
   ErrorHandler errorHandler, DTDHandler dtdHandler,
   EntityResolver entityResolver);
  public SAXOutputter(ContentHandler contentHandler,
   ErrorHandler errorHandler, DTDHandler dtdHandler,
   EntityResolver entityResolver, LexicalHandler lexicalHandler);

  public void           setContentHandler(
   ContentHandler contentHandler);
  public ContentHandler getContentHandler();
  public void           setErrorHandler(
   ErrorHandler errorHandler);
  public ErrorHandler   getErrorHandler();
  public void           setDTDHandler(DTDHandler dtdHandler);
  public DTDHandler     getDTDHandler();
  public void           setEntityResolver(
   EntityResolver entityResolver);
  public EntityResolver getEntityResolver();
  public void           setLexicalHandler(
   LexicalHandler lexicalHandler);
  public LexicalHandler getLexicalHandler();
  public void           setDeclHandler(DeclHandler declHandler);
  public DeclHandler    getDeclHandler();

  public void    setReportNamespaceDeclarations(
   boolean declareNamespaces);
  public void    setReportDTDEvents(boolean reportDtdEvents);
  public void    setFeature(String name, boolean value)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public boolean getFeature(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public void    setProperty(String name, Object value)
   throws SAXNotRecognizedException, SAXNotSupportedException;
  public Object  getProperty(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException;

  public void output(Document document) throws JDOMException;

  protected XMLReader createParser() throws Exception;

 }

XMLOutputter

The XMLOutputter class writes a JDOM Document onto an OutputStream or Writer or into a String. Using an OutputStream is preferable to using a Writer because it allows JDOM to more accurately determine which characters need to be escaped.

package org.jdom.output;

public class XMLOutputter implements Cloneable {

  public XMLOutputter();
  public XMLOutputter(String indent);
  public XMLOutputter(String indent, boolean newlines);
  public XMLOutputter(String indent, boolean newlines,
   String encoding);
  public XMLOutputter(XMLOutputter that);

  public void setLineSeparator(String separator);
  public void setNewlines(boolean newlines);
  public void setEncoding(String encoding);
  public void setOmitEncoding(boolean omitEncoding);
  public void setOmitDeclaration(boolean omitDeclaration);
  public void setExpandEmptyElements(boolean expandEmpties);
  public void setTrimAllWhite(boolean trimAllWhite);
  public void setTextTrim(boolean textTrim);
  public void setTextNormalize(boolean textNormalize);
  public void setIndent(String indent);

  public void output(Document doc, OutputStream out)
   throws IOException;
  public void output(DocType doctype, OutputStream out)
   throws IOException;
  public void output(Element element, OutputStream out)
   throws IOException;
  public void outputElementContent(Element element,
   OutputStream out) throws IOException;
  public void output(List list, OutputStream out)
   throws IOException;
  public void output(CDATA cdata, OutputStream out)
   throws IOException;
  public void output(Text text, OutputStream out)
   throws IOException;
  public void output(Comment comment, OutputStream out)
   throws IOException;
  public void output(ProcessingInstruction pi, OutputStream out)
   throws IOException;
  public void output(EntityRef entity, OutputStream out)
   throws IOException;

  public void output(Document doc, Writer out)
   throws IOException;
  public void output(DocType doctype, Writer out)
   throws IOException;
  public void output(Element element, Writer out)
   throws IOException;
  public void outputElementContent(Element element, Writer out)
   throws IOException;
  public void output(List list, Writer out)
   throws IOException;
  public void output(CDATA cdata, Writer out)
   throws IOException;
  public void output(Text text, Writer out)
   throws IOException;
  public void output(Comment comment, Writer out)
   throws IOException;
  public void output(ProcessingInstruction pi, Writer out)
   throws IOException;
  public void output(EntityRef entity, Writer out)
   throws IOException;

  public String outputString(Document doc);
  public String outputString(DocType doctype);
  public String outputString(Element element);
  public String outputString(List list);
  public String outputString(CDATA cdata);
  public String outputString(Text text);
  public String outputString(String s);
  public String outputString(Comment comment);
  public String outputString(ProcessingInstruction pi);
  public String outputString(EntityRef entity);

  public String escapeAttributeEntities(String s);
  public String escapeElementEntities(String s);

  public int parseArgs(String args, int i);

  protected Writer makeWriter(OutputStream out)
   throws UnsupportedEncodingException;
  protected Writer makeWriter(OutputStream out, String enc)
   throws UnsupportedEncodingException;

  protected void printDeclaration(Document doc, Writer out,
   String encoding) throws IOException;
  protected void printDocType(DocType docType, Writer out)
   throws IOException;
  protected void printComment(Comment comment, Writer out)
   throws IOException;
  protected void printProcessingInstruction(
   ProcessingInstruction pi, Writer out) throws IOException;
  protected void printEntityRef(EntityRef entity, Writer out)
   throws IOException;
  protected void printCDATA(CDATA cdata, Writer out)
   throws IOException;
  protected void printText(Text text, Writer out)
   throws IOException;
  protected void printString(String s, Writer out)
   throws IOException;
  protected void printElement(Element element, Writer out,
   int level,
   XMLOutputter.NamespaceStack namespaces) throws IOException;
  protected void printContentRange(List content, int start,
   int end, Writer out, int level, XMLOutputter.NamespaceStack namespaces) throws IOException;
  protected void printTextRange(List content, int start, int end,
   Writer out) throws IOException;
  protected void printAttributes(List attributes, Element parent,
   Writer out, XMLOutputter.NamespaceStack namespaces)
   throws IOException;
  protected void newline(Writer out) throws IOException;
  protected void indent(Writer out, int level)
   throws IOException;

  protected XMLOutputter.NamespaceStack createNamespaceStack();

  public Object clone();
  public String toString();

 }

XMLOutputter.NamespaceStack

The inner class NamespaceStack is used by XMLOutputter to keep track of which namespaces are in scope at any given point, and thus do not need to be redeclared. You only need to use this class if you’re subclassing XMLOutputter.

package org.jdom.output;

protected class XMLOutputter.NamespaceStack
 extends NamespaceStack {

  protected XMLOutputter.NamespaceStack();

 }

org.jdom.transform

The org.jdom.transform package allows JDOM programs to interface with TrAX-based XSLT processors by providing JDOM implementations of the TrAX Source and Result interfaces.

JDOMResult

JDOMResult can be used as an output for TrAX transformations that produce JDOM Documents.

package org.jdom.transform;

public class JDOMResult extends SAXResult {

  public static final String JDOM_FEATURE;

  public JDOMResult();

  public void        setDocument(Document document);
  public Document    getDocument();
  public void        setFactory(JDOMFactory factory);
  public JDOMFactory getFactory();
  public void        setHandler(ContentHandler handler);
  public void        setLexicalHandler(LexicalHandler handler);

 }

JDOMSource

JDOMSource can be used as an input for TrAX transformations that transform JDOM Documents.

package org.jdom.transform;

public class JDOMSource extends SAXSource {

  public static final String JDOM_FEATURE;

  public JDOMSource(Document source);

  public void      setDocument(Document source);
  public Document  getDocument();
  public void      setInputSource(InputSource inputSource)
   throws UnsupportedOperationException;
  public void      setXMLReader(XMLReader reader)
   throws UnsupportedOperationException;
  public XMLReader getXMLReader();

 }

org.jdom.xpath

The org.jdom.xpath package provides a simple interface to XPath evaluation for JDOM Documents. Jaxen is the underlying implementation.

XPath

The XPath class allows you to evaluate XPath expressions relative to JDOM node objects.

package org.jdom.xpath;

public abstract class XPath implements Serializable {

  protected XPath(String expr) throws JDOMException;

  public static XPath newInstance(String path)
   throws JDOMException;

  public List   selectNodes(Object context)
   throws JDOMException;
  public Object selectSingleNode(Object context)
   throws JDOMException;
  public String valueOf(Object context) throws JDOMException;
  public Number numberValueOf(Object context)
   throws JDOMException;
  public void   setVariable(String name, Object value)
   throws IllegalArgumentException;
  public String getXPath();

  public static List   selectNodes(Object context, String path)
   throws JDOMException;
  public static Object selectSingleNode(Object context,
   String path) throws JDOMException;

 }
Prev | Up | Next
TrAX | Home | XMLPULL
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 04, 2002


XMLPULL
Prev Appendix A. XML APIs Quick Reference Next

XMLPULL

XMLPULL is the newest and perhaps the simplest of the APIs discussed in this book. It includes only one package which contains one factory class, one interface representing the parser, and one exception. A pull-parser works in streaming mode like a SAX parser. However, it waits for the client program to request the next event rather than pushing it to it. This section is based on version 1.0.8 of XMLPULL.

org.xmlpull.v1

The org.xmlpull.v1 package includes all three parts of the XMLPULL API: XmlPullParser, XmlPullParserFactory, and XmlPullParserException.

XmlPullParser

The XmlPullParser interface represents the parser. It contains constant fields that represent type codes and features. It declares methods to get and set features and properties, to retrieve the next token from the parser, and to extract the data from that token.

package org.xmlpull.v1;

public interface XmlPullParser  {

  public static final String   NO_NAMESPACE;

  public static final int      START_DOCUMENT;
  public static final int      END_DOCUMENT;
  public static final int      START_TAG;
  public static final int      END_TAG;
  public static final int      TEXT;
  public static final byte     CDSECT;
  public static final byte     ENTITY_REF;
  public static final byte     IGNORABLE_WHITESPACE;
  public static final byte     PROCESSING_INSTRUCTION;
  public static final int      COMMENT;
  public static final int      DOCDECL;
  public static final String[] TYPES;

  public static final String FEATURE_PROCESS_NAMESPACES;
  public static final String FEATURE_REPORT_NAMESPACE_ATTRIBUTES;
  public static final String FEATURE_PROCESS_DOCDECL;
  public static final String FEATURE_VALIDATION;

  public void    setFeature(String name, boolean state)
   throws XmlPullParserException;
  public boolean getFeature(String name);
  public void    setProperty(String name, Object value)
   throws XmlPullParserException;
  public Object  getProperty(String name);
  public void    setInput(Reader in) throws XmlPullParserException;
  public void    setInput(InputStream in, String encoding)
   throws XmlPullParserException;
  public String  getInputEncoding();
  public void    defineEntityReplacementText(String entityName,
   String replacementText) throws XmlPullParserException;

  public int     getNamespaceCount(int depth)
   throws XmlPullParserException;
  public String  getNamespacePrefix(int position)
   throws XmlPullParserException;
  public String  getNamespaceUri(int position)
   throws XmlPullParserException;
  public String  getNamespace(String prefix);
  public int     getDepth();
  public String  getPositionDescription();
  public int     getLineNumber();
  public int     getColumnNumber();
  public boolean isWhitespace() throws XmlPullParserException;
  public String  getText();
  public char[]  getTextCharacters(int holderForStartAndLength);
  public String  getNamespace();
  public String  getName();
  public String  getPrefix();
  public boolean isEmptyElementTag()
   throws XmlPullParserException;
  public int     getAttributeCount();
  public String  getAttributeNamespace(int index);
  public String  getAttributeName(int index);
  public String  getAttributePrefix(int index);
  public String  getAttributeType(int index);
  public boolean isAttributeDefault(int index);
  public String  getAttributeValue(int index);
  public String  getAttributeValue(String namespace,
   String name);
  public int     getEventType()
   throws XmlPullParserException;
  public int     next()
   throws XmlPullParserException, IOException;
  public int     nextToken()
   throws XmlPullParserException, IOException;
  public void    require(int type, String namespace, String name)
   throws XmlPullParserException, IOException;
  public String  nextText()
   throws XmlPullParserException, IOException;
  public int     nextTag()
   throws XmlPullParserException, IOException;

 }

XmlPullParserFactory

The XmlPullParserFactory class creates and configures new XmlPullParser objects in an implementation independent fashion.

package org.xmlpull.v1;

public class XmlPullParserFactory  {

  public static final String PROPERTY_NAME;

  protected Vector    parserClasses;
  protected String    parserClassesLocation;
  protected Hashtable features;

  protected XmlPullParserFactory();

  public void    setFeature(String name, boolean state)
   throws XmlPullParserException;
  public boolean getFeature(String name);
  public void    setNamespaceAware(boolean awareness);
  public boolean isNamespaceAware();
  public void    setValidating(boolean validating);
  public boolean isValidating();

  public XmlPullParser newPullParser()
   throws XmlPullParserException;

  public static XmlPullParserFactory newInstance()
   throws XmlPullParserException;
  public static XmlPullParserFactory newInstance(
   String classNames, Class context)
   throws XmlPullParserException;

 }

Exceptions and Errors

XMLPULL includes a single exception class, XmlPullParserException, that represents the various things that can go wrong while parsing an XML document. In addition, a few methods that perform I/O can also throw an IOException.

XmlPullParserException

An XmlPullParserException indicates that something has gone wrong during a parse. This may be a well-formedness error in the document such as an unclosed element or a programming error in the code.

package org.xmlpull.v1;

public class XmlPullParserException extends Exception {

  protected Throwable detail;
  protected int       row;
  protected int       column;

  public XmlPullParserException(String message);
  public XmlPullParserException(String message,
   XmlPullParser parser, Throwable detail);

  public Throwable getDetail();
  public int       getLineNumber();
  public int       getColumnNumber();
  public void      printStackTrace();

 }
Prev | Up | Next
JDOM Quick Reference | Home | Appendix B. SOAP 1.1 Schemas
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 03, 2002


Appendix B. SOAP 1.1 Schemas
Prev Next

Appendix B. SOAP 1.1 Schemas

Table of Contents

The SOAP 1.1 Envelope Schema
The SOAP 1.1 Encoding Schema
W3C® SOFTWARE NOTICE AND LICENSE

This appendix contains the complete, official schema for SOAP 1.1. This is broken up into two documents the envelope schema from http://schemas.xmlsoap.org/soap/envelope/ and the encoding schema from http://schemas.xmlsoap.org/soap/encoding/. The envelope schema declares the SOAP complex types—SOAP-ENV:Envelope, SOAP-ENV:header, SOAP-ENV:Body, etc.— that define the basic structure of a SOAP document. The encoding schema declares the data types used in a SOAP document: SOAP-ENC:int, SOAP-ENC:NMTOKENS, SOAP-ENC:gYear, SOAP-ENC:Array, etc.

Both of these schemas are Copyright 2001 by Martin Gudgin. They are derived from the official SOAP 1.2 schemas, and are subject to the fairly lenient W3C Software Licensing terms are shown in the section called “W3C® SOFTWARE NOTICE AND LICENSE”. The white space has been cleaned up a little to fit everything within the margins of the page. Otherwise these are exactly as published at schemas.xmlsoap.org.

The SOAP 1.1 Envelope Schema

<?xml version='1.0' encoding='UTF-8' ?>

<!-- Schema for the SOAP/1.1 envelope

This schema has been produced using W3C's SOAP Version 1.2 schema
found at:

     http://www.w3.org/2001/06/soap-envelope

     Copyright 2001 Martin Gudgin, Developmentor.

Changes made are the following:
- reverted namespace to http://schemas.xmlsoap.org/soap/envelope/
- reverted mustUnderstand to only allow 0 and 1 as lexical values

Original copyright:

Copyright 2001 W3C (Massachusetts Institute of Technology,
Institut National de Recherche en Informatique et en Automatique,
Keio University). All Rights Reserved.
http://www.w3.org/Consortium/Legal/

This document is governed by the W3C Software License [1] as
described in the FAQ [2].

[1]http://www.w3.org/Consortium/Legal/copyright-software-19980720
[2]http://www.w3.org/Consortium/Legal/IPR-FAQ-20000620.html#DTD
-->
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           xmlns:tns="http://schemas.xmlsoap.org/soap/envelope/"
    targetNamespace="http://schemas.xmlsoap.org/soap/envelope/" >

  <!-- Envelope, header and body -->
  <xs:element name="Envelope" type="tns:Envelope" />
  <xs:complexType name="Envelope" >
    <xs:sequence>
      <xs:element ref="tns:Header" minOccurs="0" />
      <xs:element ref="tns:Body" minOccurs="1" />
      <xs:any namespace="##other" minOccurs="0"
              maxOccurs="unbounded" processContents="lax" />
    </xs:sequence>
    <xs:anyAttribute namespace="##other" processContents="lax" />
  </xs:complexType>

  <xs:element name="Header" type="tns:Header" />
  <xs:complexType name="Header" >
    <xs:sequence>
      <xs:any namespace="##other" minOccurs="0"
              maxOccurs="unbounded" processContents="lax" />
    </xs:sequence>
    <xs:anyAttribute namespace="##other" processContents="lax" />
  </xs:complexType>

  <xs:element name="Body" type="tns:Body" />
  <xs:complexType name="Body" >
    <xs:sequence>
      <xs:any namespace="##any" minOccurs="0"
              maxOccurs="unbounded" processContents="lax" />
    </xs:sequence>
    <xs:anyAttribute namespace="##any" processContents="lax" >
      <xs:annotation>
        <xs:documentation>
          Prose in the spec does not specify that attributes are
      allowed on the Body element
        </xs:documentation>
      </xs:annotation>
    </xs:anyAttribute>
  </xs:complexType>

  <!-- Global Attributes.  The following attributes are intended
       to be usable via qualified attribute names on any complex
       type referencing them.  -->
  <xs:attribute name="mustUnderstand" default="0" >
     <xs:simpleType>
     <xs:restriction base='xs:boolean'>
       <xs:pattern value='0|1' />
     </xs:restriction>
   </xs:simpleType>
  </xs:attribute>
  <xs:attribute name="actor" type="xs:anyURI" />

  <xs:simpleType name="encodingStyle" >
    <xs:annotation>
      <xs:documentation>
        'encodingStyle' indicates any canonicalization conventions
      followed in the contents of the containing element.  For
      example, the value
      'http://schemas.xmlsoap.org/soap/encoding/' indicates the
      pattern described in SOAP specification
      </xs:documentation>
    </xs:annotation>
    <xs:list itemType="xs:anyURI" />
  </xs:simpleType>

  <xs:attributeGroup name="encodingStyle" >
    <xs:attribute name="encodingStyle" type="tns:encodingStyle"/>
  </xs:attributeGroup>

  <xs:complexType name="Fault" final="extension" >
    <xs:annotation>
      <xs:documentation>
        Fault reporting structure
      </xs:documentation>
    </xs:annotation>
    <xs:sequence>
      <xs:element name="faultcode" type="xs:QName" />
      <xs:element name="faultstring" type="xs:string" />
      <xs:element name="faultactor" type="xs:anyURI"
                  minOccurs="0" />
      <xs:element name="detail" type="tns:detail" minOccurs="0"/>
    </xs:sequence>
  </xs:complexType>

  <xs:complexType name="detail">
    <xs:sequence>
      <xs:any namespace="##any" minOccurs="0"
              maxOccurs="unbounded" processContents="lax" />
    </xs:sequence>
    <xs:anyAttribute namespace="##any" processContents="lax" />
  </xs:complexType>

</xs:schema>

Prev | Up | Next
XMLPULL | Home | The SOAP 1.1 Encoding Schema
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 04, 2002


The SOAP 1.1 Encoding Schema
Prev Appendix B. SOAP 1.1 Schemas Next

The SOAP 1.1 Encoding Schema

<?xml version='1.0' encoding='UTF-8' ?>

<!-- Schema for the SOAP/1.1 encoding

This schema has been produced using W3C's SOAP Version 1.2 schema
found at:

     http://www.w3.org/2001/06/soap-encoding

     Copyright 2001 Martin Gudgin, Developmentor.
     http://www.develop.co.uk

Changes made are the following:
- reverted namespace to http://schemas.xmlsoap.org/soap/encoding/
- reverted root to only allow 0 and 1 as lexical values

     Original copyright:

Copyright 2001 W3C (Massachusetts Institute of Technology,
Institut National de Recherche en Informatique et en Automatique,
Keio University). All Rights Reserved.
http://www.w3.org/Consortium/Legal/

     This document is governed by the W3C Software License [1] as
     described in the FAQ [2].

[1]http://www.w3.org/Consortium/Legal/copyright-software-19980720
[2]http://www.w3.org/Consortium/Legal/IPR-FAQ-20000620.html#DTD
-->
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           xmlns:tns="http://schemas.xmlsoap.org/soap/encoding/"
    targetNamespace="http://schemas.xmlsoap.org/soap/encoding/" >

 <xs:attribute name="root" default="0" >
   <xs:annotation>
     <xs:documentation>
       'root' can be used to distinguish serialization roots from
       other elements that are present in a serialization but are
       not roots of a serialized value graph
     </xs:documentation>
   </xs:annotation>
   <xs:simpleType>
     <xs:restriction base='xs:boolean'>
       <xs:pattern value='0|1' />
     </xs:restriction>
   </xs:simpleType>
 </xs:attribute>

  <xs:attributeGroup name="commonAttributes" >
    <xs:annotation>
      <xs:documentation>
        Attributes common to all elements that function as
        accessors or represent independent (multi-ref) values.
        The href attribute is intended to be used in a manner
        like CONREF.  That is, the element content should be
        empty iff the href attribute appears
      </xs:documentation>
    </xs:annotation>
    <xs:attribute name="id" type="xs:ID" />
    <xs:attribute name="href" type="xs:anyURI" />
    <xs:anyAttribute namespace="##other" processContents="lax" />
  </xs:attributeGroup>

  <!-- Global Attributes.  The following attributes are intended
       to be usable via qualified attribute names on any complex
       type referencing them. -->

  <!-- Array attributes. Needed to give the type and dimensions
       of an array's contents, and the offset for
       partially-transmitted arrays. -->

  <xs:simpleType name="arrayCoordinate" >
    <xs:restriction base="xs:string" />
  </xs:simpleType>

  <xs:attribute name="arrayType" type="xs:string" />
  <xs:attribute name="offset" type="tns:arrayCoordinate" />

  <xs:attributeGroup name="arrayAttributes" >
    <xs:attribute ref="tns:arrayType" />
    <xs:attribute ref="tns:offset" />
  </xs:attributeGroup>

  <xs:attribute name="position" type="tns:arrayCoordinate" />

  <xs:attributeGroup name="arrayMemberAttributes" >
    <xs:attribute ref="tns:position" />
  </xs:attributeGroup>

  <xs:group name="Array" >
    <xs:sequence>
      <xs:any namespace="##any" minOccurs="0"
              maxOccurs="unbounded" processContents="lax" />
    </xs:sequence>
  </xs:group>

  <xs:element name="Array" type="tns:Array" />
  <xs:complexType name="Array" >
    <xs:annotation>
      <xs:documentation>
     'Array' is a complex type for accessors identified by position
      </xs:documentation>
    </xs:annotation>
    <xs:group ref="tns:Array" minOccurs="0" />
    <xs:attributeGroup ref="tns:arrayAttributes" />
    <xs:attributeGroup ref="tns:commonAttributes" />
  </xs:complexType>

  <!-- 'Struct' is a complex type for accessors identified by
       name. Constraint: No element may be have the same name as
       any other, nor may any element have a maxOccurs > 1. -->

  <xs:element name="Struct" type="tns:Struct" />

  <xs:group name="Struct" >
    <xs:sequence>
      <xs:any namespace="##any" minOccurs="0"
              maxOccurs="unbounded" processContents="lax" />
    </xs:sequence>
  </xs:group>

  <xs:complexType name="Struct" >
    <xs:group ref="tns:Struct" minOccurs="0" />
    <xs:attributeGroup ref="tns:commonAttributes"/>
  </xs:complexType>

  <!-- 'Base64' can be used to serialize binary data using base64
       encoding as defined in RFC2045 but without the MIME line
       length limitation. -->

  <xs:simpleType name="base64" >
    <xs:restriction base="xs:base64Binary" />
  </xs:simpleType>

 <!-- Element declarations corresponding to each of the simple
      types in the XML Schemas Specification. -->

  <xs:element name="duration" type="tns:duration" />
  <xs:complexType name="duration" >
    <xs:simpleContent>
      <xs:extension base="xs:duration" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="dateTime" type="tns:dateTime" />
  <xs:complexType name="dateTime" >
    <xs:simpleContent>
      <xs:extension base="xs:dateTime" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="NOTATION" type="tns:NOTATION" />
  <xs:complexType name="NOTATION" >
    <xs:simpleContent>
      <xs:extension base="xs:QName" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="time" type="tns:time" />
  <xs:complexType name="time" >
    <xs:simpleContent>
      <xs:extension base="xs:time" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="date" type="tns:date" />
  <xs:complexType name="date" >
    <xs:simpleContent>
      <xs:extension base="xs:date" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="gYearMonth" type="tns:gYearMonth" />
  <xs:complexType name="gYearMonth" >
    <xs:simpleContent>
      <xs:extension base="xs:gYearMonth" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="gYear" type="tns:gYear" />
  <xs:complexType name="gYear" >
    <xs:simpleContent>
      <xs:extension base="xs:gYear" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="gMonthDay" type="tns:gMonthDay" />
  <xs:complexType name="gMonthDay" >
    <xs:simpleContent>
      <xs:extension base="xs:gMonthDay" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="gDay" type="tns:gDay" />
  <xs:complexType name="gDay" >
    <xs:simpleContent>
      <xs:extension base="xs:gDay" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="gMonth" type="tns:gMonth" />
  <xs:complexType name="gMonth" >
    <xs:simpleContent>
      <xs:extension base="xs:gMonth" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="boolean" type="tns:boolean" />
  <xs:complexType name="boolean" >
    <xs:simpleContent>
      <xs:extension base="xs:boolean" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="base64Binary" type="tns:base64Binary" />
  <xs:complexType name="base64Binary" >
    <xs:simpleContent>
      <xs:extension base="xs:base64Binary" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="hexBinary" type="tns:hexBinary" />
  <xs:complexType name="hexBinary" >
    <xs:simpleContent>
     <xs:extension base="xs:hexBinary" >
       <xs:attributeGroup ref="tns:commonAttributes" />
     </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="float" type="tns:float" />
  <xs:complexType name="float" >
    <xs:simpleContent>
      <xs:extension base="xs:float" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="double" type="tns:double" />
  <xs:complexType name="double" >
    <xs:simpleContent>
      <xs:extension base="xs:double" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="anyURI" type="tns:anyURI" />
  <xs:complexType name="anyURI" >
    <xs:simpleContent>
      <xs:extension base="xs:anyURI" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="QName" type="tns:QName" />
  <xs:complexType name="QName" >
    <xs:simpleContent>
      <xs:extension base="xs:QName" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="string" type="tns:string" />
  <xs:complexType name="string" >
    <xs:simpleContent>
      <xs:extension base="xs:string" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="normalizedString"
              type="tns:normalizedString" />
  <xs:complexType name="normalizedString" >
    <xs:simpleContent>
      <xs:extension base="xs:normalizedString" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="token" type="tns:token" />
  <xs:complexType name="token" >
    <xs:simpleContent>
      <xs:extension base="xs:token" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="language" type="tns:language" />
  <xs:complexType name="language" >
    <xs:simpleContent>
      <xs:extension base="xs:language" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="Name" type="tns:Name" />
  <xs:complexType name="Name" >
    <xs:simpleContent>
      <xs:extension base="xs:Name" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="NMTOKEN" type="tns:NMTOKEN" />
  <xs:complexType name="NMTOKEN" >
    <xs:simpleContent>
      <xs:extension base="xs:NMTOKEN" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="NCName" type="tns:NCName" />
  <xs:complexType name="NCName" >
    <xs:simpleContent>
      <xs:extension base="xs:NCName" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="NMTOKENS" type="tns:NMTOKENS" />
  <xs:complexType name="NMTOKENS" >
    <xs:simpleContent>
      <xs:extension base="xs:NMTOKENS" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="ID" type="tns:ID" />
  <xs:complexType name="ID" >
    <xs:simpleContent>
      <xs:extension base="xs:ID" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="IDREF" type="tns:IDREF" />
  <xs:complexType name="IDREF" >
    <xs:simpleContent>
      <xs:extension base="xs:IDREF" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="ENTITY" type="tns:ENTITY" />
  <xs:complexType name="ENTITY" >
    <xs:simpleContent>
      <xs:extension base="xs:ENTITY" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="IDREFS" type="tns:IDREFS" />
  <xs:complexType name="IDREFS" >
    <xs:simpleContent>
      <xs:extension base="xs:IDREFS" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="ENTITIES" type="tns:ENTITIES" />
  <xs:complexType name="ENTITIES" >
    <xs:simpleContent>
      <xs:extension base="xs:ENTITIES" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="decimal" type="tns:decimal" />
  <xs:complexType name="decimal" >
    <xs:simpleContent>
      <xs:extension base="xs:decimal" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="integer" type="tns:integer" />
  <xs:complexType name="integer" >
    <xs:simpleContent>
      <xs:extension base="xs:integer" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="nonPositiveInteger"
              type="tns:nonPositiveInteger" />
  <xs:complexType name="nonPositiveInteger" >
    <xs:simpleContent>
      <xs:extension base="xs:nonPositiveInteger" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="negativeInteger" type="tns:negativeInteger"/>
  <xs:complexType name="negativeInteger" >
    <xs:simpleContent>
      <xs:extension base="xs:negativeInteger" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="long" type="tns:long" />
  <xs:complexType name="long" >
    <xs:simpleContent>
      <xs:extension base="xs:long" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="int" type="tns:int" />
  <xs:complexType name="int" >
    <xs:simpleContent>
      <xs:extension base="xs:int" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="short" type="tns:short" />
  <xs:complexType name="short" >
    <xs:simpleContent>
      <xs:extension base="xs:short" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="byte" type="tns:byte" />
  <xs:complexType name="byte" >
    <xs:simpleContent>
      <xs:extension base="xs:byte" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="nonNegativeInteger"
              type="tns:nonNegativeInteger" />
  <xs:complexType name="nonNegativeInteger" >
    <xs:simpleContent>
      <xs:extension base="xs:nonNegativeInteger" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="unsignedLong" type="tns:unsignedLong" />
  <xs:complexType name="unsignedLong" >
    <xs:simpleContent>
      <xs:extension base="xs:unsignedLong" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="unsignedInt" type="tns:unsignedInt" />
  <xs:complexType name="unsignedInt" >
    <xs:simpleContent>
      <xs:extension base="xs:unsignedInt" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="unsignedShort" type="tns:unsignedShort" />
  <xs:complexType name="unsignedShort" >
    <xs:simpleContent>
      <xs:extension base="xs:unsignedShort" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="unsignedByte" type="tns:unsignedByte" />
  <xs:complexType name="unsignedByte" >
    <xs:simpleContent>
      <xs:extension base="xs:unsignedByte" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="positiveInteger" type="tns:positiveInteger"/>
  <xs:complexType name="positiveInteger" >
    <xs:simpleContent>
      <xs:extension base="xs:positiveInteger" >
        <xs:attributeGroup ref="tns:commonAttributes" />
      </xs:extension>
    </xs:simpleContent>
  </xs:complexType>

  <xs:element name="anyType" />
</xs:schema>

Prev | Up | Next
Appendix B. SOAP 1.1 Schemas | Home | W3C® SOFTWARE NOTICE AND LICENSE
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 28, 2002


W3C® SOFTWARE NOTICE AND LICENSE
Prev Appendix B. SOAP 1.1 Schemas Next

W3C® SOFTWARE NOTICE AND LICENSE

Copyright © 1994-2001 World Wide Web Consortium, (Massachusetts Institute of Technology, Institut National de Recherche en Informatique et en Automatique, Keio University). All Rights Reserved. http://www.w3.org/Consortium/Legal/

This W3C work (including software, documents, or other related items) is being provided by the copyright holders under the following license. By obtaining, using and/or copying this work, you (the licensee) agree that you have read, understood, and will comply with the following terms and conditions:

Permission to use, copy, modify, and distribute this software and its documentation, with or without modification, for any purpose and without fee or royalty is hereby granted, provided that you include the following on ALL copies of the software and documentation or portions thereof, including modifications, that you make:

  1. The full text of this NOTICE in a location viewable to users of the redistributed or derivative work.

  2. Any pre-existing intellectual property disclaimers, notices, or terms and conditions. If none exist, a short notice of the following form (hypertext is preferred, text is permitted) should be used within the body of any redistributed or derivative code: "Copyright © [$date-of-software] World Wide Web Consortium, (Massachusetts Institute of Technology, Institut National de Recherche en Informatique et en Automatique, Keio University). All Rights Reserved. http://www.w3.org/Consortium/Legal/"

  3. Notice of any changes or modifications to the W3C files, including the date changes were made. (We recommend you provide URIs to the location from which the code is derived.)

THIS SOFTWARE AND DOCUMENTATION IS PROVIDED "AS IS," AND COPYRIGHT HOLDERS MAKE NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO, WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE OR DOCUMENTATION WILL NOT INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS.

COPYRIGHT HOLDERS WILL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF ANY USE OF THE SOFTWARE OR DOCUMENTATION.

The name and trademarks of copyright holders may NOT be used in advertising or publicity pertaining to the software without specific, written prior permission. Title to copyright in this software and any associated documentation will at all times remain with copyright holders.

Prev | Up | Next
The SOAP 1.1 Encoding Schema | Home | Recommended Reading
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 25, 2001


Recommended Reading
Prev Next

Recommended Reading

Books

In all likelihood, this is neither the first nor the last book, you'll read on the subject of XML and/or Java. What follows are some books that were useful to me while writing this book, and that may be helpful to you.

Callaway, Dustin R. Inside Servlets, 2d ed. Boston: Addison-Wesley, 2001; ISBN 0-201-70906-6

Harold, Elliotte Rusty. Java I/O Cambridge: O'Reilly & Associates 1999; ISBN 1-56592-485-1

Harold, Elliotte Rusty. XML Bible, 2d ed. New York: Hungry Minds, 2001; ISBN 0-7645-4760-7

Harold, Elliotte Rusty. XML Bible, Gold ed. New York: Hungry Minds, 2001; ISBN 0-7645-4819-0

Harold, Elliotte Rusty and W. Scott Means. XML in a Nutshell, 2d ed. Cambridge: O'Reilly & Associates, 2002; ISBN 0-5960-0292-0

Walsh, Norman and Leonard Muellner. DocBook: The Definitive Guide. Cambridge: O'Reilly & Associates, 2002

Specifications

What follows are the official specifications for the technologies discussed in this book such as DOM, SAX, and XML. I would not in general recommend these as the first place to look to figure out how to do something, but when you need to know exactly why something has failed, these are the canonical sources.

Biron, Paul V., and Ashok Malhotra, eds. XML Schema Part 2: Datatypes. World Wide Web Consortium, 2 May 2001.

Box, Don, David Ehnebuske, Gopal Kakivaya, Andrew Layman, Noah Mendelsohn, Henrik Frystyk Nielsen, Satish Thatte, and Dave Winer Simple Object Access Protocol (SOAP) 1.1 World Wide Web Consortium, 8 May 2000

Bray, Tim, and Jean Paoli, C. M. Sperberg-McQueen, and Eve Maler, eds. Extensible Markup Language (XML) 1.0 Second Edition. World Wide Web Consortium, 6 October 2000.

Bray, Tim, and Dave Hollander, and Andrew Layman, eds. Namespaces in XML. World Wide Web Consortium, 14 January 1999.

Clark, James XSL Transformations (XSLT) Version 1.0 World Wide Web Consortium, 16 November, 1999

Clark, James, and Steve DeRose, eds. XML Path Language (XPath) Version 1.0 World Wide Web Consortium, 16 November, 1999

Cowan, John and Richard Tobin, eds. XML Information Set. World Wide Web Consortium, 24 October 2001

DeRose, Steve, Eve Maler, and David Orchard, eds. XML Linking Language (XLink) Version 1.0 World Wide Web Consortium, 27 June 2001

Fallside, David C., ed. XML Schema Part 0: Primer W3C XML Schema Working Group World Wide Web Consortium, 2 May 2001

Hunter, Jason and Brett McLaughlin. JDOM

Kesselman, Joe, Jonathan Robie, Mike Champion, Peter Sharpe, Vidur Apparao, and Lauren Wood, eds. Document Object Model (DOM) Level 2 Traversal and Range Specification Version 1.0 W3C DOM Working Group World Wide Web Consortium, 13 November, 2000

Le Hors, Arnaud Philippe Le Hégaret, and Lauren Wood, eds. Document Object Model (DOM) Level 2 Core Specification Version 1.0 World Wide Web Consortium, 3 November, 2000

Le Hors, Arnaud, and Laurence Cable, eds. Document Object Model (DOM) Level 2 Views Specification Version 1.0 World Wide Web Consortium, 13 November, 2000

Le Hors, Arnaud, Philippe Le Hégaret, and Lauren Wood, eds. Document Object Model (DOM) Level 3 Core Specification World Wide Web Consortium, 9 April 2002

Pixley, Tom, ed. Document Object Model (DOM) Level 2 Events Specification Version 1.0 World Wide Web Consortium, 13 November, 2000

Megginson, David and David Brownell. SAX

Thompson, Henry S., David Beech Murray Maloney, and Noah Mendelson, eds. XML Schema Part 1: Structures World Wide Web Consortium, 2 May 2001

Whitmer, Ray, ed. Document Object Model (DOM) Level 3 XPath Specification World Wide Web Consortium, 28 March, 2002

Wilson Chris, Philippe Le Hégaret, and Vidur Apparao, eds. Document Object Model (DOM) Level 2 Style Specification Version 1.0 World Wide Web Consortium, 13 November, 2000

Winer, Dave. XML-RPC Specification UserLand Software, 12 December 1999

Prev | Up | Next
W3C® SOFTWARE NOTICE AND LICENSE | Home | Index
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 04, 2002


Chapter 1. XML for Data
Prev Next

Chapter 1. XML for Data

Table of Contents

Motivating XML
A Thought Experiment
Robustness
Extensibility
Ease of Use
XML Syntax
XML Documents
XML Applications
Elements and Tags
Text
Attributes
XML Declaration
Comments
Processing Instructions
Entities
Namespaces
Validity
DTDs
Schemas
Schematron
The Last Mile
Style sheets
CSS
Associating Style Sheets with XML Documents
XSL
Summary

XML was designed to be “SGML for the Web”. It was meant for the same sorts of narrative documents SGML and HTML had been used for previously: articles, books, short stories, poems, technical manuals, web pages, and so forth. Much to its inventors’ surprise, it achieved its first great successes not in the publishing and writing arenas it was intended for, but rather in the much more prosaic world of data formats. XML was enthusiastically adopted by programmers who needed a robust, extensible, standard format for data. For the most part, this was not narrative data like stories and articles, but record oriented data such as that found in databases. Uses included object serialization, financial records, vector graphics, remote procedure calls, and similar tasks. This chapter explores some of the flaws in traditional formats for such data and elucidates the features of XML that make it surprisingly well-suited for such tasks.

Motivating XML

If you’re reading this book you’re a developer. (At least I hope you are. Otherwise a lot of what I say isn’t going to make any sense :-) ) Doubtless over the course of your career you’ve written numerous programs that read and write files. And every time you wrote a new program you had to invent or learn a new file format. File formats I’ve personally had to deal with over the years include RTF, Word .doc files, tab delimited text, FITS, PDF, PostScript, and many more. You’ve probably encountered a few of these yourself. Doubtless, you’ve also seen many other formats.

If you’re like me you’ve learned to dread encountering a new file format. If it’s documented at all, the documentation is likely incomplete or worse yet misleading. Important details like byte order and line ending conventions are often left unspecified. Different tools that all claim to read and write the same format actually produce subtly different variants that are often incompatible in practice. When you think you’ve finally wrestled the last bug out of your code, you discover a file written by somebody else’s software that you can’t read; and you realize you’ve made one too many assumptions about the format, so you have to go back to the drawing board.

Consequently, when designing new file formats, developers have tended to gravitate toward the simplest formats they can imagine, often tab delimited text or comma separated values. Nonetheless, even these plain, undecorated formats often present unexpected problems. For example, should two tabs in a row be interpreted as the empty string, null, or the same as one tab? In fact, all three variations are used in practice. Java’s StringTokenizer class takes the last interpretation, two consecutive tabs are the same as one tab, even though this is the least common approach in actual data files, a fact which has surprised many Java programmers and led to not a few bugs in Java programs.[1]

A Thought Experiment

With all that in mind, let’s do a thought experiment. Imagine you’ve been tasked with writing a server side program that accepts orders over the Internet for an e-commerce site. The web server must send each completed order to the internal system, one order at a time. You’re responsible for writing the code on the server that sends the order to the internal system and for writing the code on the internal system that receives and processes the order. The only connection between the two systems is a TCP/IP network; that is, you don’t have some sort of higher level API like JDBC that lets you move data between the two systems. You need to invent a data format you can generate on one end and parse on the other end that’s flexible enough to contain all the information in a typical order. This includes the customer name, the product ordered, its price, the manufacturer’s stock keeping unit (SKU) number, the address to ship to, the tax, and the shipping and handling charges. One possibility is to place each piece of information on a separate line as shown in Example 1.1:

Example 1.1. A plain text document indicating an order for 12 Birdsong Clocks, SKU 244

c32
Chez Fred
Birdsong Clock
244
12
USD
21.95
135 Airline Highway
Narragansett
RI
02882
USD
263.40
7.0
USD
18.44
USPS
USD
8.95
USD
290.79

An alternative is to use a more complex and verbose XML format such as Example 1.2:

Example 1.2. An XML document indicating an order for 12 Birdsong Clocks, SKU 244

<?xml version="1.0" encoding="ISO-8859-1"?>
<Order>
  <Customer id="c32">Chez Fred</Customer>
  <Product>
    <Name>Birdsong Clock</Name>
    <SKU>244</SKU>
    <Quantity>12</Quantity>
    <Price currency="USD">21.95</Price >
  </Product>
  <ShipTo>
    <Street>135 Airline Highway</Street >
    <City>Narragansett</City> <State>RI</State> <Zip>02882</Zip>
  </ShipTo>
  <Subtotal currency='USD'>263.40</Subtotal>
  <Tax rate="7.0"
       currency='USD'>18.44</Tax>
  <Shipping  method="USPS" currency='USD'>8.95</Shipping>
  <Total currency='USD' >290.79</Total>
</Order>

Would you rather write the code to send and receive orders that are formatted as nice, simple linefeed delimited files as shown in Example 1.1 or as complex, marked up XML documents such as Example 1.2? Both documents contain the same information. Most uninitiated developers prefer the first, simpler form. After all each piece of information is presented on a line by itself with no extraneous markup characters getting in the way. It’s my goal to convince you that contrary to most developers’ first intuition the second form is more robust, more extensible, and much easier to work with.

Robustness

Let’s consider robustness first. Suppose your program receives the order in Example 1.3:

Example 1.3. A document indicating an order for 12 Birdsong Clocks, SKU 244?

c32
Chez Fred
Birdsong Clock
12
244
USD
21.95
135 Airline Highway
Narragansett
RI
02882
USD
263.40
7.0
USD
18.44
USPS
USD
290.79
USD
8.95

Look’s the same as Example 1.1 doesn’t it? However, if you compare it very carefully with Example 1.3 you may notice that the 12 and the 244 have changed places. What used to be an order for 12 bird clocks may now be an order for 244 whoopee cushions. Maybe somebody will notice the problem before the order is shipped and maybe they won’t. Worse yet, the shipping charge and the total price got flipped around. This entire order now costs eight dollars and ninety-five cents. Again, maybe someone will notice the problem before it’s too late and maybe not. These sorts of problems aren’t theoretical. More than one e-commerce site has lost both revenue and customer goodwill by mispricing items.

In the XML version, this simply would not be an issue because each datum is marked up with what it means. You can freely reorder the quantity and the SKU or the shipping cost and the total price without any confusion about which is which. Example 1.4 demonstrates. What can be devastating mistakes in a traditional system are harmless in XML.

Example 1.4. Still an order for 12 Birdsong Clocks, SKU 244

<?xml version="1.0" encoding="ISO-8859-1"?>
<Order>
  <Customer id="c32">Chez Fred</Customer>
  <Product>
    <Name>Birdsong Clock</Name>
    <Quantity>12</Quantity>
    <SKU>244</SKU>
    <Price currency="USD">21.95</Price >
  </Product>
  <ShipTo>
    <Street>135 Airline Highway</Street >
    <City>Narragansett</City> <State>RI</State> <Zip>02882</Zip>
  </ShipTo>
  <Subtotal currency='USD'>263.40</Subtotal>
  <Tax rate="7.0"
       currency='USD'>18.44</Tax>
  <Total currency='USD' >290.79</Total>
  <Shipping  method="USPS" currency='USD'>8.95</Shipping>
</Order>

Some readers will be objecting at this point that you would never let a mistake like that through your system. After all you check every value for sensibility. You look up the SKU in the company database to make sure it matches the product name and price before completing an order. You check every return value from a method call to see if it’s null and you catch every exception. You write extensive tests to verify that each method is doing what you think it’s doing. You use a source code control system so you can always back out changes, and you never check code in until it’s passed all the regression tests. Every line of code is scrupulously documented. In fact, you write more documentation than actual code. And you’ve never, ever missed church on Sunday. In this case your name is Donald Knuth. The rest of us need a little more help making sure we don’t do something stupid.

Even if you are that conscientious, are you really willing to gamble on everyone else who sends or receives data from you being equally anal retentive? Wouldn’t it make more sense to use the most robust format possible so that when the inevitable errors do creep in, they’ll do less damage?

Of course, XML has a lot to offer the anal developer as well. When defining constraints such as “Every order must have a shipping address”, “the currency must be one of the three letter codes USD, CAN, or GBP” or “the total cost must be the sum of the unit price times the number of items, the tax, and the shipping”, it’s easiest to use a declarative language that specifies what the constraints are without elaborating the actual code to check these constraints. When your data is XML, you can use a declarative schema language to define and test such constraints. Indeed, you have a choice of several schema languages. The simplest and most broadly supported, the classic document type definition (DTD), allows you to verify that all required elements are present in the required order with any necessary attributes. The W3C XML schema language goes further and lets you constrain the contents of particular elements and attributes so that you can guarantee that the total price is a decimal number greater than 1.00. Schematron, the most powerful schema language of all, allows you to state multi-element constraints such as “the actual price must be less than or equal to the suggested retail price”. I’ll discuss all of these languages in more detail later in this chapter and the rest of the book. For now what you need to know is that you can list all the constraints on a document in a simple fashion and check those constraints without writing a lot of extra code to do so. You feed your documents through a validator before you act on them. Validation becomes a separate, modular and more maintainable part of the process. You can even change constraints or add new ones without recompiling your code.

Extensibility

Robustness isn’t the only advantage of the XML approach. The XML solution is also far more extensible. For example, suppose you suddenly discover a need to add a discount percentage to some products. The change to the XML is straightforward. Just add an extra element:

  <Product>
    <Name>Birdsong Clock</Name>
    <Quantity>12</Quantity>
    <SKU>244</SKU>
    <Price currency="USD">21.95</Price >
    <Discount>.10</Discount>
  </Product>

The change to the plain text file (or the equivalent binary file) is much less obvious. You can certainly add an extra line of data. However, then everything that follows it will be out of order. You could put the new information at the end of the document, but then it isn’t close to the item it logically belongs with. And suppose not all orders have discounts. Will there be blank lines for products that don’t have discounts? How will your program recognize that it’s supposed to convert an empty string into a zero discount rather than NaN or throwing an exception? This is not an insurmountable problem, but the simple solution is becoming more complex.

Now suppose someone wants to add a gift message field whose value can contain line breaks. Now the data can contain the delimiter character! You can probably escape the line breaks as \n or some such, and then escape the backslash character as \\, but your nice simple solution is becoming quite a bit more complex. However, once again this is not a problem for XML as this solution demonstrates:

  <GiftMessage>
     Happy Birthday Monica!

    Love Always,
    Tracy
  </GiftMessage>

Throughout this example, I’ve assumed that each order is for exactly one product. That’s probably not true. Some customers will order multiple products at a time. Thus each order will contain between one and an indefinite number of products. Different products may even be going to different addresses. Do you break each individual item into a separate order document and repeat the customer information? If so how do you calculate the total shipping and total cost? Or do you allow multiple products in a single order? If so how do you tell where one product ends and the next begins? Again, none of these problems are unsolvable, but the simple solution proves more and more complex as the needs grow. The XML approach, by contrast, scales very well to expanded functionality in a very obvious way. Example 1.5 is an XML document that accomplishes all of the above. The boundaries between the individual parts are obvious.

Example 1.5. An XML document indicating an order for multiple products shipped to multiple addresses

<?xml version="1.0" encoding="ISO-8859-1"?>
<Order>
  <Customer id="c32">Chez Fred</Customer>
  <Product>
    <Name>Birdsong Clock</Name>
    <SKU>244</SKU>
    <Quantity>12</Quantity>
    <Price currency="USD">21.95</Price >
    <ShipTo>
      <Street>135 Airline Highway</Street >
      <City>Narragansett</City> <State>RI</State> <Zip>02882</Zip>
    </ShipTo>
  </Product>
  <Product>
    <Name>Brass Ship's Bell</Name>
    <SKU>258</SKU>
    <Quantity>1</Quantity>
    <Price currency="USD">144.95</Price >
    <Discount>.10</Discount>
    <ShipTo>
      <GiftRecipient>Samuel Johnson</GiftRecipient>
      <Street>271 Old Homestead Way</Street >
      <City>Woonsocket</City> <State>RI</State> <Zip>02895</Zip>
    </ShipTo>
    <GiftMessage>
      Happy Father's Day to a great Dad!

      Love,
      Sam and Beatrice
    </GiftMessage>
  </Product>
  <Subtotal currency='USD'>393.85</Subtotal>
  <Tax rate="7.0"
       currency='USD'>28.20</Tax>
  <Shipping  method="USPS" currency='USD'>8.95</Shipping>
  <Total currency='USD' >431.00</Total>
</Order>

This example still isn’t really complete. Many pieces are missing including the credit card information, billing address, and more. Real world examples are larger and more complex than can comfortably fit in a book. Adding these other parts would only stretch the flat format further and make the advantages of XML still more obvious. The more complex your data is, the more important it is to use a hierarchical format like XML rather than a flat format like tab or line-delimited text.

Ease of Use

Now here’s the real kicker: not only is the XML document far more robust. Not only is it much more extensible in the face of both expected and unexpected changes. Not only does it more easily adapt to more complex structures. It is also easier for your programs to read! Writing a program to accept orders written in XML will be many times easier than writing a program to accept orders delivered in simple line delimited files. “How can that be?” you may be asking. After all, the program reading the XML document has to hunt for less than signs and quotation marks rather than just picking each piece of data off of a line. It has to make sure not to confuse any less than signs and quotation marks that may appear in the data itself with those in the markup. It has to deal with data that may extend across multiple lines. And in fact, there are many more possibilities not evident in this simple example that a real program has to handle.

Fortunately none of this matters to you as a developer because you don’t have to do any of it. Instead of writing the code to process XML documents directly, you let an XML parser do the hard work for you. A parser is a software library that knows how to read XML documents and handle all the markup it finds. The parser takes responsibility for checking documents for well-formedness and validity. Your own code reads the XML document only through the parser’s API. At this level, you can simply ask the parser to tell you what it saw in any particular element. Or you can ask the parser to tell you everything it sees as soon as it sees it. In either case, the parser just gives you the data after resolving all the markup. For instance, if you want to ask the parser what the total price was, it can tell you 290.79 and that this price has the currency USD. You don’t have to concern yourself with stripping off the markup around the information you want. Nor do you necessarily have to take the information in the order it appears in the input document. If you want the total price before the customer name, you can have it. If you just want to look at the price and ignore the rest of the order completely, you can do that too. You take the information in the form that’s convenient to you without worrying excessively about low level serialization details.

Note

One of the original ten goals for XML was that “It shall be easy to write programs which process XML documents.” Originally, this was interpreted as meaning that a “Desperate Perl Hacker” could write an XML parser in a weekend. Later it became clear that XML was simply too complex, even in its simplest form, for this goal to be met. However, the understanding of this requirement changed to mean that a typical programmer could use any of a number of free tools and libraries to process XML. Given this interpretation, the goal has most certainly been met.

The parser shields you from a lot of irrelevant details that you don’t really care about. These include:

  • How text is encoded: in Unicode, ASCII, Latin-1, SJIS, or something else

  • Whether carriage returns, line feeds, or both separate lines

  • How reserved characters such as < are escaped when used in the plain text parts of the document

  • Whether the byte order is big-endian or little-endian

None of these issues actually matter. None of them have any effect on what the data means or what the format allows you to say. However, when designing a data format, you must answer all these questions. As soon as you’ve said, “The underlying format of the data is XML”, every one of these questions is answered. Some are answered by simply choosing one possible solution. (The less than sign is escaped as &lt;.) Others are answered by allowing all possibilities and letting the parser sort things out (line endings). In all cases, the design problem is greatly simplified by picking XML as the underlying format.

[1] This interpretation makes sense once you realize that java.util.StringTokenizer is designed for parsing Java source code, not for reading tab delimited data files. Nonetheless many programmers do use it for reading tab delimited data.

Prev | Up | Next
Acknowledgements | Home | XML Syntax
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 21, 2002


XML Syntax
Prev Chapter 1. XML for Data Next

XML Syntax

This is not an introductory book about XML. I certainly expect that you have some experience with XML documents before now. Nonetheless, when writing programs to process XML it’s even more important to make sure that you are totally crystal clear about the exact terminology used when discussing XML. Therefore I’d like to take a few pages to briefly review the proper terminology for discussing XML, as well as to clarify a few points that are often confused or misunderstood.

XML Documents

The precise meaning of “XML document” is defined by the XML 1.0 specification published by the Worldwide Web Consortium (W3C). This specification provides a detailed BNF grammar defining exactly what is and is not an XML document. Anything that satisfies the document production in that BNF grammar and adheres to the fifteen well-formedness constraints is an XML document.[2] Anything that does not is not an XML document.

Well-formedness is the minimum requirement for an XML document. A document that is not well-formed is not an XML document. Parsers cannot read it. A parser is not allowed to fix a malformed document. It cannot take a best-guess at what the document author intended. When a parser encounters a malformed document, it stops parsing and reports the error. It will not read any further in the document. [3] Depending on which API you’re accessing the parser through, you may or may not have already received some information from the parts of the document before the error. However, under no circumstances will the parser give you any data from after the first well-formedness error in the document.

The detailed rules an XML document must follow aren’t so important here since the parser will check them for you. Very roughly an XML document must have a single root element. All start-tags must be matched by end-tags. All attribute values must be quoted. And only the Unicode characters that are legal in XML may be used in the document. (Almost all Unicode characters are legal in XML documents. The only ones really ruled out are the C0 controls like null, bell, and form feed.)

Note

Occasionally developers ask how they can parse a document that is almost, but not quite a well-formed XML document. For example, it may end with a form feed inserted by some Unix text editor to separate documents. Or perhaps it’s part of an infinite stream of elements, the last of which is never seen so there’s no end-tag for the root element. Imagine, for example, weather observations or stock quotes being pushed across the Internet as XML elements.

The short answer is that you can’t parse these things because they are not XML documents, even if they use a lot of tags and attributes and other XML-like markup. The long answer is that you may be able to write a non-XML-aware program to preprocess the streams, fix up any well-formedness mistakes you see, and only then pass the fixed documents to the XML parser. However, the XML parser must receive a complete well-formed document. It cannot work with anything less.

There’s another way to look at XML documents besides simply as a sequence of characters that adheres to certain rules, and it’s one that sometimes makes sense, especially when writing programs that process XML documents. An XML document is a tree. It has a root node that contains various child nodes. Some of these child nodes have children of their own. Others are leaf nodes that have no children.

There are roughly five different kinds of nodes in an XML tree:

root

Also known as the document node, this is the abstract node that contains the entire XML document. Its children include comments, processing instructions, and the root element of the document.

element

An XML element with a name, a list of attributes, a list of in-scope namespaces, and a list of children.

text

The parsed character data between two tags (or any other kind of non-text node).

comment

An XML comment such as <!-- This needs to be fixed. -->. The contents of the comment are its data. A comment does not have any children.

processing instruction

A processing instruction such as <?xml-stylesheet type="text/css" href="order.css"?> A processing instruction has a target and a value. It does not have any children.

Depending on context, some details of this tree structure can be understood differently. For example, some tree models consider parsed entities or CDATA sections to be additional kinds of nodes. Others simply merge them into the tree structure as elements and text nodes. Some models allow one text node to follow another. Others require each text node to be the maximum contiguous run of text not interrupted by some other kind of node. Some models include the document type declaration and/or the XML declaration as a node. Others ignore them. Probably the most hotly debated point is how to handle attributes and namespaces. I chose to not consider them as nodes in the tree in their own right, treating them instead as properties of elements. Generally even those tree models such as XPath that do treat them as separate nodes still don’t make them children of the element they belong to. For now the details aren’t too important. The broad outline is the same for pretty much all the tree models.

Caution

There’s some argument about whether it really makes sense to talk about an XML document as having any independent existence separate from the text that makes up the document. After all, the XML 1.0 specification only defines concepts like document and element in terms of text strings. Later W3C specifications like the XML Information Set (Infoset) and the Document Object Model (DOM) do suggest a more abstract understanding of the components of an XML document. However, these specifications are much more controversial than XML 1.0 itself, and not as broadly implemented or accepted. For the purposes of writing programs that process XML, I do find it useful to consider XML documents more abstractly; and I will do so in this book. However, even here there’s a split depending on which API you choose. DOM is a very abstract model of XML documents that defines classes representing elements, attributes, comments, and more. SAX defines almost no such classes, however. It presents the content of an XML document almost exclusively as strings and arrays of characters.

XML Applications

An XML application is a specific XML vocabulary that contains particular elements and attributes. It is not a software program that somehow uses XML like the EditML Pro XML editor or the Mozilla web browser. XML applications limit the very flexible rules of XML to a finite set of elements of certain types. For example, DocBook is an XML application designed for technical manuscripts such as the book you’re reading now. Elements it defines include book, chapter, para, sect1, sect2, programlisting, and several hundred others. When writing a DocBook document, you have to use these elements; and you have to use them in certain ways. For instance, a sect2 element can be a child of a sect1 but not a child of a sect3 or a chapter. Scalable Vector Graphics (SVG) is an XML application for line art. Elements it defines include line, circle, ellipse, polygon, polyline, and so forth. All SVG documents are XML documents, but not all XML documents are SVG documents.

An XML application can have a schema that defines what is and is not a legal document for that application. Schemas can be written in a variety of languages including Document Type Definitions (DTDs), the W3C XML Schema Language, RELAX NG, Schematron, and numerous others. Depending on the power of the schema language used, it may also be necessary to specify additional rules for the application in less-formal prose. For example, the XHTML 1.1 specification includes the requirement that “There must be a DOCTYPE declaration in the document prior to the root element. If present, the public identifier included in the DOCTYPE declaration must reference the DTD found in Appendix C using its Formal Public Identifier.” None of the common schema languages allow you to require anything about the DOCTYPE declaration.

An instance document is an instance of an XML application, whether formally defined or not. That is, it is an XML document with a root element and whatever other content it possesses that satisfies all the rules of some XML application. There are many possible instance documents for any one XML application, just as there are many programs that can be written in any one programming language.

Elements and Tags

The fundamental unit of XML is the element. You can write good XML documents without using any other XML construct. If for some reason you have a grudge against comments, processing instructions, attributes, or namespaces, you can pretend they don’t exist and still write well-formed XML documents. However, you must use elements. Every XML document has at least one element. You cannot write XML documents without using elements.

Logically every element has four key pieces:

  • A name

  • The attributes of the element

  • The namespaces in scope on the element

  • The content of the element

In addition, once schemas become more prevalent and parsers and APIs are revised to support them, it may also make sense to talk about the element’s type. For now, though, there’s not a lot of practical help to be gained by considering the type.

Furthermore, DOM and XPath also have mutually incompatible concepts of the value of an element. However, in both cases, the value is derived purely from the element content, so it’s not really a separate thing.

Syntactically, in the text form of an XML document, elements are delimited by tags. Start-tags begin with a < immediately followed by the element name. End-tags begin with a </ immediately followed by the element name. Both start and end-tags terminate with >. Everything in between the two tags is the content of the element. For example, this is a Quantity element with the content “12”:

<Quantity>12</Quantity>

Tags and elements are closely related, but they are not the same thing. Be wary of books that confuse them. An element is the whole sandwich including bread, meat, cheese, pickles, and mayonnaise, while the tags are just the bread. An element is composed of a start-tag, followed by content, followed by an end-tag.

It is possible that an element may have no content. In this case it is called an empty element. For example, this is an empty Quantity element:

<Quantity></Quantity>

The start-tag butts right up against the end-tag. There is not even a single space character between them. By contrast, this next element is not empty because it does contain some white space, even if it doesn’t contain anything else:

<Quantity> </Quantity>

Besides start-tags and end-tags, there is one other kind of tag, the empty-element tag. An empty-element tag begins with a < followed by an element name like a start-tag. However, it ends with a />. For example, this is an empty Quantity tag:

<Quantity/>

This tag both starts and ends a Quantity element. The content of this element is nothing, just like the content of <Quantity></Quantity>. Indeed <Quantity/> is just syntax sugar for <Quantity></Quantity>. They mean exactly the same thing. No application should treat these two constructs as different in any way. Indeed, most XML parsers and APIs won’t even tell you which form the element took in the source document. In both cases, what’s reported is an empty element with the name “Quantity”. How that element was represented is not important.

As well as text, an element can also contain one or more child elements. These are elements that are completely contained between the element’s start-tag and end-tag, and are not contained inside any other element also contained in the parent element. For example, this ShipTo element has four child elements: Street, City, State, and Zip:

  <ShipTo>
    <Street>135 Airline Highway</Street >
    <City>Narragansett</City> <State>RI</State> <Zip>02882</Zip>
  </ShipTo>

In addition to the four child elements, this ShipTo element also contains some white space; for example, the single space character between </City> and <State>. These spaces form text nodes that are also counted among the element’s children. Text nodes like these that are composed of nothing but white space are sometimes called ignorable white space. This is an unfortunate turn of phrase. Sometimes you can ignore these nodes, but most of the time you can’t. The more proper term is white space in element content.[4]

All the elements contained inside an element are called the element’s descendants. Only the highest level are the children. The descendants include not only the children, but the children of the children, the children of the children’s children, and so forth. If you look at Example 1.2 again, you’ll see that the Order element has 15 descendant elements.

An element can also have mixed content. This is when an element contains both child elements and text nodes containing non-whitespace characters. For example, this variant ShipTo element has both the child elements you saw before as well as text nodes containing the strings “Chez Fred” and “Apt. 17D”:

  <ShipTo>
    Chez Fred
    <Street>135 Airline Highway</Street >
    Apt. 17D
    <City>Narragansett</City> <State>RI</State> <Zip>02882</Zip>
  </ShipTo>

Mixed content is very useful, indeed almost essential, for XML applications that contain narratives such as books and stories. Such applications include XHTML, DocBook, TEI, and XSL Formatting Objects. Mixed content is much less useful and much more cumbersome for data-oriented applications. XML documents that are intended for computers to read, as opposed to XML documents that are intended for humans to read, should use mixed content sparingly, if at all.

Text

XML documents are text. Each XML document is a sequence of characters. These characters are taken from the Unicode character set. However, XML documents can be written in any character set which your XML parser knows how to convert to Unicode, providing that it is properly specified in the document’s encoding declaration in the XML declaration.

Caution

Many developers have decided that they can make XML more efficient by defining a binary version. This tends to be based on some vague notion that binary formats are inherently smaller or faster than text formats. These developers rarely have any actual evidence to back up this claim, which is not surprising since it isn’t true. XML documents are routinely smaller and faster to read than the equivalent binary files in standard applications like Oracle, Microsoft Word, Microsoft Excel, and so forth. The fact is modern binary file formats are quite bloated, but disks have gotten so large that almost no one’s noticed or cared. Nonetheless, there seems to be a large pool of programmers who mistakenly believe:

  1. File size matters.

  2. They can compress better than gzip.

  3. Human legible/human editable data doesn’t matter.

All three beliefs have been empirically proven false time and time again. Nonetheless, about once a month some developer somewhere announces that they’ve come up with yet another special purpose binary compression format for XML. These have proven completely pointless in practice. There is no actual benefit to these formats, and no one needs one. Worse yet, such a format substantially eliminates many of the existing benefits of XML.

Unicode is a character set with room for over one million different characters, though currently (Unicode 3.2) a few less than 100,000 are defined. Scripts covered by Unicode include Latin, Cyrillic, Greek, Hebrew, Arabic, Devanagari, the Han ideographs, and many more.

Caution

Contrary to what you may have heard, Unicode is not a two-byte character set and really never has been. Since there are more than a million different spaces for characters in Unicode, an arbitrary Unicode character cannot be represented by a single two-byte unsigned integer such as Java’s char data type. Prior to Unicode 3.1 all defined Unicode characters had code points less than 65,536, which fooled some developers into thinking they could get away with using two-byte chars. However, it’s long been known that more than 65,536 characters are actually used on Earth today and that Unicode would have to assign characters outside the Basic Multilingual Plane (the first 65,535 characters) to accommodate them. Although characters were not actually assigned code points greater than 65,536 until Unicode 3.1, the space for them has long been reserved. XML was designed by forward-thinkers who saw the problems ahead, and prepared for the eventual expansion of Unicode. Consequently XML documents can use the full range of all million-plus characters available in Unicode. Java’s designers weren’t as prescient though, and restricted the char data type to two-bytes. Consequently Java programmers need to go through some pretty nasty gyrations to adequately handle Unicode documents (including XML documents).

With a very few exceptions any character defined in Unicode can be used in the text content of an element or the value of an attribute. In brief, the exceptions are:

The C0 controls

The non-printing characters such as null and formfeed, between code points 0 and 31 (decimal). The carriage return, linefeed, and the horizontal tab are allowed.

The surrogate blocks

The surrogate blocks are two sets of 1024 code points each, which are used to extend Unicode beyond the Basic Multilingual Plane by allowing some characters to be represented as two surrogate characters. You can include surrogate pairs in an XML document in an encoding like UTF-16 that uses surrogate pairs. You just can’t treat an individual half of a surrogate pair as a character by itself.

The byte order mark

The byte order mark, also known as the zero-width non-breaking space, can be used at the beginning of a document to indicate the encoding and endianness of the document, but cannot be used elsewhere in the document.

All other characters are fair game, including some you probably shouldn’t be using anyway such as characters in the private use area and compatibility characters Unicode offers purely for interoperability with existing character sets.

The rules for characters used in the names of things (elements, attributes, entities, etc.) are a little stricter. In brief, only letters, digits, and ideographs defined in Unicode 2.0 can be used. In addition the punctuation marks -, ., _, and : are also legal. Digits, the hyphen, and the period cannot be the first character in a name. Other punctuation marks as well as new characters first defined in Unicode 3.0 and later are not allowed anywhere in a name. These are essentially the same rules used for naming variables, methods, and classes in Java. The major difference is that XML allows the hyphen and Java doesn’t (it’s reserved for the minus sign) while Java allows the dollar sign and XML doesn’t. XML also allows the colon, unlike Java. However, XML reserves this for use with namespaces. It should not be used as an arbitrary name character.

XML parsers faithfully preserve white space. A string containing only white space is not the same as a string containing nothing at all. A string with leading and trailing white space is not the same as the equivalent string with white space trimmed. Some specific XML applications may decide that white space is not significant in certain contexts. However, in generic XML all white space is significant and must be accounted for.

Attributes

Attributes are name value pairs associated with elements. The name of an attribute may be any legal XML name. The value may be any string of text, even potentially including characters like < and ". The document author needs to escape such characters as &lt; and &quot;. However, the parser will resolve these references before passing the data to your application. The attribute value is enclosed in either single or double quotes, and the name is separated from the value by an equals sign. For example, this Subtotal element has a currency attribute with the value USD:

<Subtotal currency='USD'>393.85</Subtotal>

The quote marks are not part of the attribute value. Whether single or double quotes are used or whether there’s extra white space around the equals sign is not important. Most parsers don’t bother to report the difference. These two elements are also the same as the previous one:

<Subtotal currency="USD">393.85</Subtotal>
<Subtotal currency = "USD">393.85</Subtotal>

Attributes are unordered. There is no difference between these two elements:

<Tax rate="7.0" currency="USD">27.57</Tax>
<Tax currency="USD" rate="7.0">27.57</Tax>

When a parser tells you which attributes are attached to an element, it may or may not provide them in the same order they had in the input document. Some APIs report the attributes using an unordered data structure like a hash table. Others use an array or a list, but even in these cases there’s no guarantee that the order of the attributes in the list matches the order of the attributes in the start-tag.

Perhaps most surprisingly, attribute values whose type is not CDATA are normalized. This means that all leading and trailing white space is stripped from the value, and runs of white space characters are compressed to a single space. This does not apply to any of the attributes in the examples seen so far because untyped attributes are not normalized. However, once you add a DTD it is possible to declare that an attribute has type ID, IDREF, IDREFS, NMTOKEN, and several other types. Attributes of these types are always normalized before being passed to the client application.

Note

Tim Bray, one of the primary authors of XML 1.0, has admitted that normalization of attribute values was a mistake. In his words, “Why the $#%%!@! should attribute values be ‘normalized’ anyhow? This was a pure process failure: at no point during the 18-month development cycle of XML 1.0 did anyone stand up and say ‘why are you doing this?’ I’d bet big bucks that if someone had, the silly thing would have died a well-deserved death.” [5]

XML Declaration

Most XML documents begin with an XML declaration. An XML declaration has a version attribute with the value 1.0 and may have optional standalone and encoding attributes. For example, this XML declaration says that the document is written in XML 1.0 in the ISO-8859-1 (Latin-1) character set and does not require the parser to read the external DTD subset:

<?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?>

The version attribute always has the value 1.0. If XML 1.0 is ever revised, this may change to some other value. As I write this, there’s a hotly debated proposal at the W3C for a new version of XML code named “Blueberry” which would make XML marginally more compatible with Unicode 3.0 and later as well as making it easier to edit with some brain damaged IBM mainframe software that can’t handle files where lines end in carriage returns, line feeds, or both. If this gets adopted (and I for one hope it doesn’t) this may lead to a new value for the version attribute. However, for now, version is effectively fixed with the value 1.0.

The encoding attribute identifies the character set and encoding in which the document is written. Whatever the encoding is, one of the jobs of the parser is to convert the document to Unicode before passing it to the client application. Most APIs don’t offer any means of finding out what the original encoding was. You’ll simply receive Unicode strings from which all traces of the original encoding have been removed.

The standalone attribute specifies whether the XML parser may have to read parts of the DTD that are outside the instance document to correctly parse the file. This is mostly a hint for the parser. Some parser APIs may tell you what the value was, but you generally don’t need to worry about it. The parser either will or won’t read external entities as necessary. By the time your code gets hold of the document, all of this will have already been taken care of. You need not concern yourself with it.

Comments

XML comments are almost exactly like HTML comments. They begin with <!-- and end with -->. For example, here’s a comment you might find in an order document:

<!-- Please make sure this order goes out ASAP! -->

Everything between the <!-- and the --> should be ignored. In fact, most parsers and APIs do make the comments available to you if you want them, mostly so you can round trip documents (read them in and then write them back out again with everything still intact). However, beyond this use case, you really shouldn’t pay much attention to comments in your programs. Some HTML systems abuse comments to support server side includes or editor specific extensions. Since XML is much more flexible than HTML, however, you can use elements, attributes, or, as a last resort, processing instructions for these use cases.

Processing Instructions

Processing instructions are used to tell particular software how it should handle an XML document after the document has been parsed. Generally, processing instructions are used for meta-information that may apply to documents from many different domains and XML vocabularies. For instance, the most common processing instruction, xml-stylesheet, tells a browser or other formatter where it can find the stylesheet it should apply to the document. This can be used with DocBook documents, XHTML documents, Human Resources Markup Language documents, or the custom XML application you invented last Tuesday to catalog your baseball card collection. For another example, the Apache XML Project’s Cocoon application server reads cocoon-process processing instructions to figure out what processes to apply to a document before sending it to a user. This processing instruction tells Cocoon to replace the XInclude include elements with the contents of the documents they reference:

<?cocoon-process type="xinclude"?>

The basic syntax of a processing instruction is <?, followed immediately by an XML name identifying the target of the processing instruction, followed by white space and any data at all, followed by ?>.

Unlike elements or attributes, processing instructions can be added to a document without considering whether or not the DTD or schema allows it. Most schema languages do not consider the presence, absence, or structure of processing instructions when determining validity. Furthermore, unlike elements, processing instructions can appear before, after, or inside the root element. They are frequently placed in the document prolog, though they can appear in the document body or after the root element as well.

Most of the time, the processing instruction is not associated with any one XML application. For instance, an XML application may describe gene sequences, 16th century Italian love poetry, financial records, or vector graphics. However, each of these might need to be loaded into a Web browser which would apply a stylesheet to it. Processing instructions can be inserted into a document to support this without changing or affecting the normal document structure. In essence, processing instructions provide an out-of-band channel for passing information to software other than the program that would normally read a document.

XML parsers report the target and contents of processing instructions to the client application. However, they provide no further support for interpreting the data in the processing instruction. For instance, many processing instructions use a pseudo-attribute format like this:

<?xml-stylesheet type="text/xml" href="limited.xsl"?>

However, as far as the XML parser is concerned, the data in this processing instruction is just a string that happens to contain some equals signs and quotation marks. These are not treated differently than any other character.[6] Both the syntax and semantics of the data is completely up to the application reading the document. Processing instructions are specifically for information that is not related to XML.

Entities

XML documents are not necessarily the same thing as XML files. A single XML document may be composed of several different files. Indeed, the pieces that make up an XML document may not be files at all, but may instead be records in a database, data sent out over the Internet by a web server in response to a CGI query, a small part of a much larger file, or something stranger still.

The individual storage units that make up any one XML document are called entities. Every XML document has at least one entity, the document entity. This is the storage unit, be it a file or something else, that holds the root element of the document. Every other entity in a document has a name. There are five such kinds of named entities, and they are classified according to three criteria:

Internal or external

The replacement text of an internal entity is defined as a string literal in the document’s DTD. The replacement text of an external entity is read out of a different file located via a URL.

Parsed or unparsed

A parsed entity contains XML. It is itself well-formed, and may even be a complete XML document if it has a root element. (Some entities that are only intended to be used as parts of other documents do not have root elements). You can think of a parsed entity as something that will be pasted right into the middle of an XML document, such that the resulting document would still be well-formed.

An unparsed entity can contain anything at all, including binary data. Unparsed entities are not pasted (even metaphorically) into XML documents. Instead a URL to the entity’s data is provided in an ENTITY declaration in the DTD. Then this entity is referenced in an attribute with the type ENTITY or ENTITIES in the document. An unparsed entity also has a notation that defines the type of the data in the unparsed entity (e.g. GIF image or C source code). Like the URL, the notation is also specified in the DTD rather than in the instance document. In practice, unparsed entities and notations are not much used.

General or parameter

A general entity is used within the instance document. A general entity reference begins with an &. A parameter entity is used within the DTD. A parameter entity reference begins with a %. Since this book focuses on processing instance documents, we’ll consider general entities primarily.

Not all combinations are possible. In fact, there are exactly five kinds of named entities:

Internal parsed general entities

The familiar entity references like &amp; and &copy; that are defined completely in the DTD. For example, this declaration defines the copy entity as the text “Copyright”:

<!ENTITY copy "Copyright">

These entities are used in element content and attribute values.

External parsed general entities

External parsed general entities are just like internal parsed general entities except that their replacement text is read from a separate document rather than the DTD. The document is identified by a relative or absolute URL. For example, this declaration define defines the legal entity as the content read from the URL http://www.example.com/legal.xml:

<!ENTITY legal SYSTEM "http://www.example.com/legal.xml">

The file such an entity is read from is just like another XML document except that it has a text declaration instead of an XML declaration, may not have a document type declaration, and might not have a single root element.

External unparsed general entities

External unparsed general entities refer to files containing non-XML, binary data. They are declared similarly to external parsed entities, but they also have a notation. For example, these definitions identify an unparsed entity named logo at the URL http://www.example.com/logo.png with the notation image/png:

<!NOTATION PNG SYSTEM "image/png">
<!ENTITY logo SYSTEM "http://www.example.com/logo.png" NDATA PNG>

Unparsed entities are referenced by attributes with type ENTITY or ENTITIES rather than by entity references. For example, such an attribute might be declared like this:

<!ELEMENT figure EMPTY>
<!ATTLIST figure logo ENTITY #REQUIRED>

Instances of the figure element would look like this:

<figure source="logo"/>

The parser does not actually provide you with the contents of an unparsed entity. Instead it tells you the URI from which the data can be retrieved and the notation for that data. However, you have to use Java’s networking and I/O classes to get the data at that URI.

Internal parsed parameter entities

Internal parsed parameter entities are used purely within the DTD. The replacement text is provided by a string literal in the DTD. References to these entities begin with a percent sign. They’re often used to parameterize content models and attribute types. For example, the DocBook DTD defines the intermod.redecl.module parameter entity as the word IGNORE:

<!ENTITY % intermod.redecl.module "IGNORE">

Unlike a general entity reference, the %intermod.redecl.module; parameter entity reference can only be used in the DTD, not in the instance document. Since our focus is on instance documents, not DTDs, you won’t see a lot of these in this book.

External parsed parameter entities

External parsed parameter entities are used purely with the DTD. The replacement text is provided by a DTD fragment at a given URL. References to these entities begin with a percent sign. They often connect the different parts of a modular DTD into one coherent whole. For example, the DocBook DTD defines the dbpool parameter entity using a PUBLIC ID that loads the DTD fragment at the relative URL dbpoolx.mod:

<!ENTITY % dbpool PUBLIC
"-//OASIS//ELEMENTS DocBook XML Information Pool V4.1.2//EN"
"dbpoolx.mod">

Again, since our focus is on instance documents and not DTDs, you won’t see a lot of these in this book.

Namespaces

Namespaces are not part of XML 1.0. Namespaces were invented about a year after XML 1.0 was released to help sort out the rapidly expanding world of XML applications that all needed to be mixed together in the same documents. There are many good XML applications that don’t use them at all. For example, DocBook 4.1.2, the XML application in which this book was written, is completely namespace free as are XML-RPC and RSS 0.9.1. However, even if you can write very useful XML applications without thinking about namespaces, you’re going to encounter namespaces when you work with XML applications designed by other developers. Consequently it’s important to have a solid understanding of them.

The key idea of namespaces is that each element is bound to a Uniform Resource Identifier (URI) (a URL in practice). If IBM only uses URIs in the ibm.com domain and Sun only uses URIs in the sun.com domain, then there won’t be any confusion between Sun’s Book element and IBM’s Book element, even if they’re used in the same document. Just look at the URIs to tell which is which.

Note

A URI identifies a resource, but it does not necessarily locate it. URIs include not only Uniform Resource Locators (URLs) but also Uniform Resource Names (URNs). For instance, a URN for this book based on its ISBN number is urn:isbn:0201771861; but this does not tell you where you can find a copy of the book. However, most developers agree that only absolute URLs should be used as namespace URIs, and most XML applications follow this suggestion.

The URIs are purely string identifiers. Even if the URI is a URL, the parser does not connect to the server and try to download the document that’s found there. Indeed there may not be any such document. When plugged into web browsers, namespace URLs often produce 404 Not Found errors. You can use namespaces in standalone systems without any network connection at all. You don’t even have to have access to DNS. For the same reason, two different URLs that point to the same page define two different namespaces. For example, http://ns.cafeconleche.org/Orders/, http://ns.cafeconleche.org/Orders, and http://ns.cafeconleche.org/Orders/index.html are one page but three different namespaces.

Since URIs contain many characters which are illegal in element names as well as being excessively long to type, short prefixes stand in for the URIs. The prefixes are separated from the local name by a colon. For instance, instead of the URI http://www.w3.org/2001/XInclude you might use the prefix xinclude or xi. An include element in the http://www.w3.org/2001/XInclude namespace would then be written as xi:include. This element has the prefix xi, the local name include, the qualified name xi:include, and the namespace URI http://www.w3.org/2001/XInclude.

xmlns:prefix attributes bind particular prefixes to particular URIs within the element where the attribute appears. For example, inside this Order element, the prefix xi is bound to the URI http://www.w3.org/2001/XInclude:

<Order xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:include href="order_details.xml"/>
</Order>

Each prefix used in an element or attribute name must be bound to a URI. Failure to do this is a namespace well-formedness error. Although you can parse documents without considering namespaces, in practice most parsers and APIs check namespaces by default and a violation of namespace well-formedness is as serious as a violation of the rules of XML 1.0.

The prefix can change as long as the URI stays the same. For example, this element is the same as the previous one:

<Order xmlns:xinclude="http://www.w3.org/2001/XInclude">
  <xinclude:include href="order_details.xml"/>
</Order>

You can also define a default namespace that applies to elements without prefixes. For example, Example 1.6 places the Order element and all its descendants in the http://ns.cafeconleche.org/Orders/ namespace, even though none of them have prefixes.

Example 1.6. An XML document that uses a default namespace

<?xml version="1.0" encoding="ISO-8859-1"?>
<Order xmlns="http://ns.cafeconleche.org/Orders/">
  <Customer id="c32">Chez Fred</Customer>
  <Product>
    <Name>Birdsong Clock</Name>
    <SKU>244</SKU>
    <Quantity>12</Quantity>
    <Price currency="USD">21.95</Price >
    <ShipTo>
      <Street>135 Airline Highway</Street >
      <City>Narragansett</City> <State>RI</State> <Zip>02882</Zip>
    </ShipTo>
  </Product>
  <Subtotal currency='USD'>263.405</Subtotal>
  <Tax rate="7.0"
       currency='USD'>18.44</Tax>
  <Shipping  method="USPS" currency='USD'>8.95</Shipping>
  <Total currency='USD' >290.79</Total>
</Order>

Although it’s most common to place the namespace binding attributes on the root element, they can appear on other elements deeper in the hierarchy. They can even override previous bindings in the ancestor elements. This is especially common with the binding of the default namespace. For instance, in Example 1.7 the Order, Customer, Product, Name, SKU, Price, Subtotal, Tax, Shipping, and Total elements are all in the http://ns.cafeconleche.org/Orders/ namespace. However, the ShipTo, Street, City, State, and Zip elements are in the http://ns.cafeconleche.org/Address/ namespace.

Example 1.7. An XML document that uses two default namespaces

<?xml version="1.0" encoding="ISO-8859-1"?>
<Order xmlns="http://ns.cafeconleche.org/Orders/">
  <Customer id="c32">Chez Fred</Customer>
  <Product>
    <Name>Birdsong Clock</Name>
    <SKU>244</SKU>
    <Quantity>12</Quantity>
    <Price currency="USD">21.95</Price >
    <ShipTo xmlns="http://ns.cafeconleche.org/Address/">
      <Street>135 Airline Highway</Street >
      <City>Narragansett</City> <State>RI</State> <Zip>02882</Zip>
    </ShipTo>
  </Product>
  <Subtotal currency='USD'>263.40</Subtotal>
  <Tax rate="7.0"
       currency='USD'>18.44</Tax>
  <Shipping  method="USPS" currency='USD'>8.95</Shipping>
  <Total currency='USD' >290.79</Total>
</Order>

Although it’s less common, prefixes can also be attached to attribute names to indicate what namespace the attribute is in. For example, XLink uses this to distinguish between the XLink attributes such as type and href and attributes with the same names that might be used in elements that need to become XLinks. This ShipTo element is also a simple XLink to the recipient’s e-mail address:

    <ShipTo xmlns="http://ns.cafeconleche.org/Address/"
         xmlns:xlink="http://www.w3.org/1999/xlink"
         xlink:type="simple" xlink:href="mailto:chezfred@yahoo.com"
    >
      <GiftRecipient>Samuel Johnson</GiftRecipient>
      <Street>271 Old Homestead Way</Street >
      <City>Woonsocket</City> <State>RI</State> <Zip>02895</Zip>
    </ShipTo>

Unprefixed attributes are never in any namespace. Unlike elements, they cannot be in the default namespace. Furthermore, they are not in the same namespace as the element to which they are attached. If an attribute does not have a prefix, it is not in a namespace.

On occasion namespace prefixes are used in attribute values, element content, and even in processing instructions. In these cases the nearest ancestor element that contains a binding for that prefix establishes what URI the prefix is mapped to. Inside an element with an xmlns:prefix attribute, we say that the namespace is in scope even if it isn’t obviously used anywhere in that element. Namespaces in scope on an element include not only those that the element itself declares but also those that are declared on that element’s ancestors. An element can redeclare a namespace prefix so that it’s mapped to a different URI on the element and the element’s children than in the element’s parent. Slightly more commonly, an element can change the default namespace that applies within the element and its content.

When writing software to process XML documents that use namespaces, you almost always want to make your code dependent on the URI, not the prefix. If you’re comparing two elements for equality, compare them by URI and local name, not prefix and local name. If you’re searching for an element of a certain type, look for an element with the right URI and local name, not the right prefix and local name.

[2] The well-formedness constraints specify requirements that are difficult or impossible to express in BNF form; for example, that “The Name in an element’s end-tag must match the element type in the start-tag.”

[3] A few parsers continue reading so they can report further errors after the first one. However, they only report errors, not content.

[4] Technically, whether or not white space only nodes are considered to be “white space in element content” depends on the content specification for the element given by the DTD. A white space only text node is only white space in element content when the content specification for the parent element in the DTD indicates that the parent element can only contain child elements but not mixed content. Since Example 1.2 doesn't have a DTD, this can't possibly be white space in element content.

[5] Re: Attribute normalisation and character entities, posted on the xml-dev mailing list, January 27, 2000

[6] JDOM and dom4j actually do provide special support for processing instructions written in this pseudo-attribute format. However, they both do a substantial amount of work in their own classes to support this interface, beyond what the parser provides.

Prev | Up | Next
Chapter 1. XML for Data | Home | Validity
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 22, 2002


Validity
Prev Chapter 1. XML for Data Next

Validity

Programmers have long known the value of verifiable preconditions on functions and methods. (A lot of us carelessly don’t use them, but that’s a topic for another book.) One of the important innovations of XML is the ability to place preconditions on the data the programs read, and to do this in a simple declarative way. XML allows you to say that every Order element must contain exactly one Customer element, that each Customer element must have an id attribute that contains an XML name token, that every ShipTo element must contain one or more Streets, one City, one State, and one Zip, and so forth. Checking an XML document against this list of conditions is called validation. Validation is an optional step but an important one.

There is more than one language in which you can express such conditions. Generically, these are called schema languages, and the documents that list the constraints are called schemas. Different schema languages have different strengths and weaknesses. The document type definition (DTD) is the only schema language built into most XML parsers and endorsed as a standard part of XML. However, because of the extensible nature of XML, many other schema languages have been invented that can easily be integrated with your systems.

DTDs

A DTD focuses on the element structure of a document. It says what elements a document may contain, what each element may and must contain in what order, and what attributes each element has.

Element Declarations

In order to be valid according to a DTD, each element used in the document must be declared in an ELEMENT declaration. For example, this is an ELEMENT declaration that says that Name elements contain #PCDATA, that is, text but no child elements.

<!ELEMENT Name (#PCDATA)>

Elements that can have children are declared by listing the names of their children in order, separated by commas. For example, this ELEMENT declaration says that an Order element contains a Customer element, a Product element, a Subtotal element, a Tax element, a Shipping element, and a Total element in that order:

<!ELEMENT Order (Customer, Product, Subtotal, Tax, Shipping, Total)>

The parenthesized list of things an element can contain is called the element’s content model. You can attach a question mark after an element name in the content model to indicate that the element is optional; that is, that either zero or one instance of the element may occur at that position. You can attach an asterisk after the element name to indicate that zero or more instances of the element may occur at that position, or a plus sign to indicate that one or more instances of the element must occur at that position. For example, this element declaration states that a ShipTo element must contain zero or one GiftRecipient elements, one or more Street elements, and exactly one City, State, and Zip elements in that order:

<!ELEMENT ShipTo (GiftRecipient?, Street+, City, State, Zip)>

You can use a vertical bar instead of a comma to indicate that either one or the other of the elements may appear. You can group collections of elements with parentheses to indicate that the entire group should be treated as a unit. You can suffix a *, ?, or + to the group to indicate that zero or more, zero or one, or one or more of those groups may appear at that point. Finally, you may replace the entire content model with the keyword EMPTY to specify that the element must not contain any content at all.

Attribute Declarations

A DTD also specifies which attributes may and must appear on which elements. Each attribute is declared in an ATTLIST declaration which specifies:

  • The element to which the attribute belongs

  • The name of the attribute

  • The type of the attribute

  • The default value of the attribute

For example, this ATTLIST declaration says that every Customer element must have an attribute named id with type ID:

<!ATTLIST Customer id ID #REQUIRED>

DTDs define ten different types for attributes:

CDATA

Any string of text; the default type for undeclared attributes in invalid documents

NMTOKEN

A string composed of one or more legal XML name characters. Unlike an XML name, a name token may start with a digit.

NMTOKENS

A white space separated list of name tokens

ID

An XML name that is unique among ID type attributes in the document

IDREF

An XML name used as an ID attribute value on some element in the document

IDREFS

A white space separated list of XML names used as ID attribute values somewhere in the document

ENTITY

The name of an unparsed entity declared in an ENTITY declaration in the DTD

ENTITIES

A white space separated list of unparsed entities declared in the DTD

NOTATION

The name of a notation declared in a NOTATION declaration in the DTD

Enumeration

A list of all legal values for the attribute, separated by vertical bars. Each possible value must be an XML name token.

Most parsers and APIs will tell you what the type of an attribute is if you want to know, but in practice this knowledge is not very useful. W3C XML schema language schemas offer much more complete data typing for both elements and attributes, including not only these types but also the more customary data types like int and double.

DTDs allow four possible default values for attributes:

#REQUIRED

Each element in the instance document must provide a value for this attribute.

#IMPLIED

Each element in the instance document may or may not provide a value for this attribute. If an element does not, then no default value is provided from the DTD.[7]

#FIXED "value"

The attribute always has the value that follows #FIXED in double or single quotes, whether or not it’s present in the instance document.

"value"

By default the attribute has the value specified in the DTD in single or double quotes. However, individual instances of the element may specify a different value.

Parsers may or may not tell you whether an attribute came from the instance document or was defaulted in from the DTD. It’s relatively rare that you care about this one way or the other. However, if you’re using a document that relies heavily on attribute values from DTDs, (e.g. for namespace declarations) make sure you’re using a parser that does read the external DTD subset.

Example 1.8 is a complete DTD for order documents of the type shown in this chapter. It uses both ELEMENT and ATTLIST declarations.

Example 1.8. A DTD for order documents

<!ELEMENT Order (Customer, Product+, Subtotal, Tax, Shipping, Total)>
<!ELEMENT Customer (#PCDATA)>
<!ATTLIST Customer id ID #REQUIRED>
<!ELEMENT Product (Name, SKU, Quantity, Price, Discount?,
                   ShipTo, GiftMessage?)>
<!ELEMENT Name (#PCDATA)>
<!ELEMENT SKU (#PCDATA)>
<!ELEMENT Quantity (#PCDATA)>
<!ELEMENT Price (#PCDATA)>
<!ATTLIST Price currency (USD | CAN | GBP) #REQUIRED>
<!ELEMENT Discount (#PCDATA)>
<!ELEMENT ShipTo (GiftRecipient?, Street+, City, State, Zip)>
<!ELEMENT GiftRecipient (#PCDATA)>
<!ELEMENT Street (#PCDATA)>
<!ELEMENT City   (#PCDATA)>
<!ELEMENT State  (#PCDATA)>
<!ELEMENT Zip    (#PCDATA)>
<!ELEMENT GiftMessage (#PCDATA)>
<!ELEMENT Subtotal (#PCDATA)>
<!ATTLIST Subtotal currency (USD | CAN | GBP) #REQUIRED>
<!ELEMENT Tax (#PCDATA)>
<!ATTLIST Tax currency (USD | CAN | GBP) #REQUIRED
              rate CDATA "0.0"
>

<!ELEMENT Shipping (#PCDATA)>
<!ATTLIST Shipping currency (USD | CAN | GBP) #REQUIRED
                   method   (USPS | UPS | Overnight) "UPS">
<!ELEMENT Total (#PCDATA)>
<!ATTLIST Total currency (USD | CAN | GBP) #REQUIRED>

Document Type Declarations

Documents are associated with particular DTDs using document type declarations. This is a document type declaration that points to the DTD in Example 1.8:

<!DOCTYPE Order SYSTEM "order.dtd">

The document type declaration is placed in the instance document’s prolog, after the XML declaration but before the root element start-tag. For example,

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE Order SYSTEM "order.dtd">
<Order>
  ...

This does assume that the DTD can be found in the same directory where the document itself is. If you prefer you can use an absolute URL instead. For example,

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE Order SYSTEM "http://www.ibiblio.org/xml/dtds/order.dtd">
<Order>
  ...

Even though Example 1.5 satisfies all the conditions expressed in Example 1.8, it is not valid because it does not have a document type declaration pointing to that DTD.

Caution

The acronym DTD is correctly used only to mean “document type definition”. It should never be used to mean “document type declaration”. The document type declaration may contain or point to the document type definition (or both); but it is not the same thing.

DTDs are not just about validation. They can also affect the content of the instance document itself. In particular, they can:

  • Define entities

  • Define notations

  • Provide default values for attributes

Assuming you’re using a validating parser, there is little reason to care about how such things happen. The entities the DTD defines will be resolved before you see them. The notations will be applied to the appropriate elements and entities. A default attribute value will be just one more attribute in an element’s list of attributes. Some APIs may tell you what entity a particular element came form or whether an attribute value was defaulted from the DTD or present in the instance document. However, most of the time you simply do not need to know this.

Schemas

The W3C XML Schema Language (schemas for short, though it’s hardly the only schema language) addresses several limitations of DTDs. First schemas are written in XML instance document syntax, using tags, elements, and attributes. Secondly, schemas are fully namespace aware. Thirdly, schemas can assign data types like integer and date to elements, and validate documents not only based on the element structure but also on the contents of the elements.

Example 1.9 shows a schema for order documents. Where order.dtd uses an ELEMENT declaration, order.xsd uses an xsd:element element. Where order.dtd uses an ATTLIST declaration, order.xsd uses an xsd:attribute element.

But order.xsd doesn’t just repeat the same constraints found in order.dtd. It also assigns types and ranges to the elements. For instance, it requires that all the money elements—Tax, Shipping, Subtotal, Total, and Price—contain a decimal number such as 9.85, 7.2, or -3.25. [8] If one of these elements contained text that was not a decimal number such as “France”, then the validator would notice and report the problem. DTDs cannot detect mistakes like this. A DTD can note that there is no Price element where one is expected, but it cannot determine that the Price element does not actually give a price.

Example 1.9. order.xsd: a schema for order documents

<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  <xsd:element name="Order">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="Customer">
          <xsd:complexType>
            <xsd:simpleContent>
              <xsd:extension base="xsd:string">
                <xsd:attribute name="id" type="xsd:ID"/>
              </xsd:extension>
            </xsd:simpleContent>
          </xsd:complexType>
         </xsd:element>
        <xsd:element name="Product" maxOccurs="unbounded">
          <xsd:complexType>
            <xsd:sequence>
              <xsd:element name="Name"     type="xsd:string"/>
              <xsd:element name="SKU"
               type="xsd:positiveInteger"/>
              <xsd:element name="Quantity"
               type="xsd:positiveInteger"/>
              <xsd:element name="Price"    type="MoneyType"/>
              <xsd:element name="Discount" type="xsd:decimal"
                           minOccurs="0"/>
              <xsd:element name="ShipTo">
                <xsd:complexType>
                  <xsd:sequence>
                    <xsd:element name="GiftRecipient"
                     type="xsd:string"
                     minOccurs="0" maxOccurs="unbounded"/>
                    <xsd:element name="Street"
                     type="xsd:string"/>
                    <xsd:element name="City" type="xsd:string"/>
                    <xsd:element name="State"
                     type="xsd:string"/>
                    <xsd:element name="Zip" type="xsd:string"/>
                  </xsd:sequence>
                </xsd:complexType>
              </xsd:element>
              <xsd:element name="GiftMessage" type="xsd:string"
                           minOccurs="0"/>
            </xsd:sequence>
          </xsd:complexType>
        </xsd:element>
        <xsd:element name="Subtotal" type="MoneyType"/>
        <xsd:element name="Tax">
          <xsd:complexType>
            <xsd:simpleContent>
              <xsd:extension base="MoneyType">
                <xsd:attribute name="rate" type="xsd:decimal"/>
              </xsd:extension>
            </xsd:simpleContent>
          </xsd:complexType>
        </xsd:element>
        <xsd:element name="Shipping">
          <xsd:complexType>
            <xsd:simpleContent>
              <xsd:extension base="MoneyType">
                <xsd:attribute name="method" type="xsd:string"/>
              </xsd:extension>
            </xsd:simpleContent>
          </xsd:complexType>
        </xsd:element>
        <xsd:element name="Total" type="MoneyType"/>
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>

  <xsd:complexType name="MoneyType">
    <xsd:simpleContent>
      <xsd:extension base="xsd:decimal">
        <xsd:attribute name="currency" type="xsd:string"/>
      </xsd:extension>
    </xsd:simpleContent>
  </xsd:complexType>

</xsd:schema>

There are multiple ways to indicate that a document should satisfy a known schema. The most common is an xsi:noNamespaceSchemaLocation attribute on the root element of the instance document. The xsi prefix is bound to the http://www.w3.org/2001/XMLSchema-instance URI. For example,

<?xml version="1.0" encoding="ISO-8859-1"?>
<Order xsi:noNamespaceSchemaLocation="order.xsd"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  ...

Some parsers also provide ways to specify a schema from inside a program, for instance by setting various properties. I’ll discuss this more when we get to programmatic validation in Chapter 7.

Schemas are still pretty bleeding edge technology at the time of this writing. (May, 2002). There are only a few parsers that provide incomplete implementations of the full W3C XML Schemas 1.0 specification. Nonetheless, developers have been clamoring for this functionality (if not necessarily this syntax) for some time so schemas seem likely to achieve broad adoption relatively quickly.

For the moment, schema support is limited to simple validation, much as DTD support is. A schema-aware parser will read an XML document, compare what it sees there to a schema, and return a boolean result: either the document satisfies the schema or it does not. In the event the document fails to satisfy the schema, the parser might give you a line number and a more detailed error message about exactly what the problem is, but that’s it. More complete use of schemas, in which parsers tell you what the type of any element is so you can, for example, convert elements with type xsd:int to actual Java ints, are still a matter for research and experiment.

Schematron

Rick Jelliffe’s Schematron is a radically different approach to an XML schema language. Whereas other languages are conservative (everything not permitted is forbidden) Schematron is liberal (everything not forbidden is permitted). Furthermore, Schematron is based on XPath so it can check cooccurence constraints between elements and attributes; e.g. that the content of the total price element must be equal to the sum of the content of the subtotal, tax, and shipping elements. Finally, Schematron can be implemented as an XSLT stylesheet rather than requiring special software.

Example 1.10 shows a Schematron schema for order documents. To keep the example smaller, I did not test absolutely everything I could. Instead, I took advantage of Schematron’s liberality to test only those conditions that neither DTDs nor schemas can validate; for instance, that the total price is the sum of the subtotal, the tax, and the shipping. I haven’t necessarily lost anything by doing this. I can validate a single document against multiple different kinds of schemas. For instance, orders could first be checked against the DTD, then checked against a W3C XML Schema Language schema, and only checked against this Schematron schema if they passed the first two tests.

Example 1.10. order.sct: a Schematron schema for order documents

<?xml version="1.0"?>
<schema xmlns="http://www.ascc.net/xml/schematron">
  <title>A Schematron Schema for Orders</title>
  <pattern>
    <rule context="Order">
      <!-- Due to round-off error, floating point numbers
           should rarely be compared for direct equality.
           For this purpose, it's enough if they're accurate
           within one penny. -->
      <assert test="(Shipping+Subtotal+Tax - Total)&lt;0.01
                and (Shipping+Subtotal+Tax - Total)&gt;-0.01">
        The subtotal, tax, and shipping
        must add up to the total.
      </assert>

      <assert test=
       "(Subtotal+Shipping)*((Tax/@rate) div 100.0)
         - Tax &lt; 0.01 and (Subtotal+Shipping)*((Tax/@rate)
         div 100.0)-Tax &gt; -0.01
      ">
        The tax was incorrectly calculated.
      </assert>

    </rule>
  </pattern>
</schema>

XPath is not by itself Turing complete so there are still some limits to what you can express in a Schematron schema. For instance, you can’t sum up the Quantity times the Price for each Product element and make sure that equals the Subtotal. However, Schematron is still a lot more powerful than other schema languages.

Schematron is implemented in a very unusual fashion. First you run your Schematron schema through an XSLT processor using a skeleton stylesheet Jelliffe provides. This produces a new XSLT stylesheet. In essence, this compiles the Schematron schema into an XSLT stylesheet. The compiler is itself written in XSLT. You then transform all your instance documents using the compiled schema. If any of the assertions fail, the output will contain the assertion message. Otherwise it will contain just the XML declaration. For example, using Michael Kay’s SAXON XSLT processor, to validate Example 1.2 against Example 1.10:

C:\XMLJAVA>saxon order.sct skeleton1-5.xsl>order_sct.xsl
C:\XMLJAVA>saxon order.xml order_sct.xsl
<?xml version="1.0" encoding="utf-8"?>

Schematron is the idiosyncratic product of one person. It is therefore not a standard part of any major parsers, unlike DTDs and the W3C XML Schema Language. However, it’s not particularly difficult to install Jelliffe’s Schematron validation software into most systems. Since Schematron is implemented in XSLT, all you need is a good API to access an XSLT engine. I’ll take this up again in the final chapter when I discuss APIs for XSLT.

The Last Mile

Schematron is powerful, but there are still some checks it cannot perform. In particular, it cannot perform any checks that require information external to the document and the schema. For example, it cannot verify that the page at a referenced URL is reachable. It cannot verify that a file exists on the local file system. It cannot compare the SKUs, names, and prices in an order document with their values in a remote database. None of the extant schema languages allow you to state conditions like these.

Java can do all of these things. The java.net.URL class can easily test whether a URL is live. The exists() method of the java.io.File class is a simple test for whether a file is where you think it is. JDBC is a whole API remote database access. However, unlike the more limited constraints of DTDs, the W3C XML Schema Language, or even Schematron, simply listing the conditions is not enough. To test such conditions, you have to write the code that tests them. Nobody’s done the hard work for you. There will always be some constraints you need a full-blown programming language to check. Indeed doing exactly this will be a major focus of the remainder of this book.

One thing you can learn from the existing languages is the clean way they separate validation from processing. If you design your own validation layer, you should do that too. Perform all validation before the document is processed for its contents. If possible, separate the constraints from the code that checks them.

[7] This is really a bad choice of terminology. Nothing is being implied here. A more accurate keyword would be #OPTIONAL. However #IMPLIED is what XML gives us.

[8] It would be possible to go further and require that each money item be a positive number with two decimal digits of precision such as 9.85 but not 7.2 or -3.25, but for now I wanted to keep this example smaller.

Prev | Up | Next
XML Syntax | Home | Style sheets
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 21, 2002


Style sheets
Prev Chapter 1. XML for Data Next

Style sheets

For the most part this book is going to focus on XML documents used as input to and output from various kinds of programs. In many cases it’s entirely possible that the XML documents will be both written and read by software and that no human being ever even looks at the documents. However, on occasion people do need to load XML documents into a browser or print them on paper so they can read them. For this purpose XML documents are rather coarse, especially for non-programmers. To pretty them up, you can attach a style sheet to the document that specifies how each element should be presented. There are two main languages used for this purpose today, Cascading Style Sheets (CSS) and the Extensible Stylesheet Language (XSL).

CSS

CSS is a very straightforward, non-XML, declarative language. CSS rules attach style properties to elements. Each rule has a selector specifying which elements it applies to. The simplest selector is merely an element name such as Order or Price. This is followed by a pair of braces containing the style properties to apply to the selected elements. Each property has a name such as font-weight or display and a value that’s appropriate for that property. The name and value are separated by a colon. For example, this rule says that the Customer element should be bold faced:

Customer {font-weight: bold}

CSS rules often set multiple properties for a single element. Individual properties are separated by semicolons. For example, this rule says that the Order element (and all its descendants) should have the font-family serif and the font-size 16 points.

Order {font-family: serif; font-size: 16pt}

Most of the properties set on an element such as Order are inherited by all its descendants such as Customer, Price, and State. However, if a descendant element sets a different value for an inherited property, then that value overrides the inherited value. For example, this rule sets the font-family for the gift message to ZapfChancery or any script font if ZapfChancery is not available on the local system. It overrides the choice of a serif font that GiftMessage inherits from its ancestor Order element.

GiftMessage {font-family: ZapfChancery, script}

The selector syntax can be adjusted to apply to multiple elements at one time by separating the names by commas. For example, this rule specifies that five different elements all have the value block for the display property. This means each will be separated from the previous and following elements by a line break.

Street, Subtotal, Tax, Shipping, Total {display: block}

CSS also allows you to select elements according to attributes, parentage, siblings, ID, link status, and more. An asterisk can be used to stand in for any element.

Example 1.11 is a complete CSS stylesheet for order documents such as Example 1.2. It adds several new features including setting the display property to none to hide the SKU element and using the :before selector and the content property to add a little boiler-plate text in front of several elements. However, although useful, these facilities are limited. You still can’t reorder the elements, and the content property is limited to plain text, no markup.

Example 1.11. A CSS stylesheet for order documents

Order {font-family: serif; font-size: 16pt;
       display: block; line-height: 20pt;
       margin-left: 0.25in
      }
ShipTo {margin-left: 0.5in; display: block }
ShipTo:before {content: "Ship to:"; margin-left: -0.25in }
Product {font-size: 12pt; display: block }
Customer {font-weight: bold; display: block }
GiftMessage {font-family: ZapfChancery, script}
Street, Subtotal, Tax, Shipping, Total {display: block}
Quantity:before {content: "Quantity: "}
SKU {display: none}
Subtotal:before {content: "Subtotal: $"}
Price:before {content: "Unit Price: $"}
Tax:before {content: "Tax: $"}
Shipping:before {content: "Shipping: $"}
Total:before {content: "Total: $"}

Associating Style Sheets with XML Documents

A few browsers may allow the user to specify which stylesheet to apply to a document. In general, however, an XML document that will be read by people must carry an xml-stylesheet processing instruction in its prolog that indicates which stylesheet should be applied to that document. This processing instruction has two pseudo-attributes, type and href. The type pseudo-attribute identifies the MIME media type of the stylesheet, text/css for Cascading Style Sheets. The href pseudo-attribute specifies the relative or absolute URL where the style sheet can be found. For example, this xml-stylesheet processing instruction says that a CSS style sheet named order.css can be found in the same directory where the XML document itself was found:

<?xml-stylesheet type="text/css" href="order.css"?>

Figure 1.1 shows Example 1.2 loaded into Mozilla after the stylesheet in Example 1.11 has been attached. Loaded into Opera or Netscape 6, the results would be similar. Internet Explorer 5.5 and earlier have much weaker support for CSS, and would not do nearly as good a job formatting the XML. Netscape 4.x and earlier have absolutely no support for displaying XML documents.

Figure 1.1. The clock order styled by CSS

This isn’t bad. A browser with full support for CSS Level 2 does let you do a lot, but there are still numerous issues. The recipient’s name should be included in the ship-to information. The subtotal, tax, shipping, and total dollar amounts should really be aligned. And the product name, quantity, and price should probably be in a multi-row table with one row for each item in the order. Although CSS does support tables, it requires that the markup in the XML document already be structured in a very tabular fashion, which it isn’t here.

XSL

CSS is limited to specifying how the text content of each element is styled. It presents pretty much the entire content of the XML document in pretty much the order it was present in the stylesheet. It cannot add boilerplate text such as the return policy or extract just the elements needed for an address label. It is limited to specifying the appearance of elements already present in the XML document without changing their order or content.

XSL, by contrast, can reorder elements, delete certain elements, add content that wasn’t present in the original document, combine multiple documents into a single result, and more. Because of its ability to add text to what’s in the XML document before the document is shown to the user, XSL is much more suited for many of the examples in this book that contain only the raw data without a lot of extraneous text. XSL is a much more powerful stylesheet language than CSS. However, it is also comparatively difficult to learn.

XSL is divided into two complementary parts, XSL Transformations (XSLT) and XSL Formatting Objects (XSL-FO). Unlike CSS, both XSLT and XSL-FO are XML applications. XSLT stylesheets and XSL-FO documents are well-formed XML documents.

XSLT is a Turing-complete functional programming language designed specifically for describing transformations from one XML format into another. (A functional programming language is one with no side effects, in which the order of evaluation of statements makes no difference to the final result. The classic examples of functional languages are Scheme and Lisp.) An XSLT processor reads an XML document and an XSLT stylesheet, and transforms the document according to the instructions found in the stylesheet. It then outputs the transformed document. XSLT is designed for XML-to-XML transformations, but it can transform XML to HTML, XML to plain text, or XML to any other text format such as TeX or troff.

XSL-FO is an XML application that describes the layout of the various elements on a printed page. It has elements that represent paragraphs, list items, margins, and so forth. An XSL-FO processor converts an XSL-FO document into some other format that can be printed or viewed such as PDF, TeX, SVG, or plain text. This book was actually written in an XML application called DocBook. An XSLT style sheet converted the source DocBook files into XSL-FO document, which was then further processed to produce the PDF document that was sent to the printer.

XSLT is based on the notion of templates. The XSLT processor compares nodes in an input document tree to templates in the stylesheet. When it finds a match, it follows the instructions in that template. These instructions can include XML to output, details about what to copy from the input into the output, and directions for which nodes to process next.

Example 1.12 is an XSLT stylesheet that describes how order documents can be converted into XSL Formatting Objects. This particular stylesheet formats the products as a table, and the rest of the elements as paragraphs. The products are written in a 12-point font while the rest of the document uses 16-point fonts.

Example 1.12. An XSLT stylesheet for order documents

<?xml version="1.0"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:fo="http://www.w3.org/1999/XSL/Format">

  <!-- Try to make the output look half decent -->
  <xsl:output indent="yes"/>

  <xsl:template match="Order">
    <fo:root>

      <fo:layout-master-set>
        <fo:simple-page-master master-name="only">
          <fo:region-body margin-left="0.5in"
                          margin-top="0.5in"/>
        </fo:simple-page-master>
      </fo:layout-master-set>

      <fo:page-sequence master-reference="only">

        <fo:flow flow-name="xsl-region-body">
          <xsl:apply-templates select="Customer"/>
          <xsl:apply-templates select="ShipTo"/>
          <fo:table font-size="12pt"
                    space-before="24pt" space-after="24pt">
            <fo:table-column column-width="2in"/>
            <fo:table-column column-width="1in"/>
            <fo:table-column column-width="1in"/>
            <fo:table-column column-width="1in"/>
            <fo:table-body>
              <fo:table-row font-weight="bold">
                <fo:table-cell>
                  <fo:block>Product</fo:block>
                </fo:table-cell>
                <fo:table-cell>
                  <fo:block>Quantity</fo:block>
                </fo:table-cell>
                <fo:table-cell>
                  <fo:block>Unit Price</fo:block>
                </fo:table-cell>
                <fo:table-cell>
                  <fo:block>Subtotal</fo:block>
                </fo:table-cell>
              </fo:table-row>
              <xsl:apply-templates select="Product"/>
            </fo:table-body>
          </fo:table>
          <xsl:apply-templates select="Tax"/>
          <xsl:apply-templates select="Shipping"/>
          <xsl:apply-templates select="Total"/>
        </fo:flow>

      </fo:page-sequence>

    </fo:root>
  </xsl:template>

  <xsl:template match="Customer">
    <fo:block font-size="16pt" font-family="serif"
              line-height="20pt">
       Ship to:
    </fo:block>
    <fo:block font-size="16pt" font-family="serif"
              margin-left="0.5in" line-height="20pt">
       <xsl:apply-templates/>
    </fo:block>
  </xsl:template>

  <xsl:template match="ShipTo">
    <fo:block font-size="16pt" font-family="sans-serif"
              line-height="18pt" margin-top="20pt"
              margin-left="0.5in">
      <xsl:apply-templates select="Street"/>
    </fo:block>
    <fo:block font-size="16pt" font-family="sans-serif"
              line-height="18pt" margin-left="0.5in">
      <xsl:apply-templates select="City"/>&#xA0;
      <xsl:apply-templates select="State"/>&#xA0;
      <xsl:apply-templates select="Zip"/>
    </fo:block>
  </xsl:template>

  <xsl:template match="Product">
    <fo:table-row>
      <fo:table-cell>
        <fo:block><xsl:value-of select="Name"/></fo:block>
      </fo:table-cell>
      <fo:table-cell>
        <fo:block><xsl:value-of select="Quantity"/></fo:block>
      </fo:table-cell>
      <fo:table-cell>
        <fo:block>$<xsl:value-of select="Price"/></fo:block>
      </fo:table-cell>
      <fo:table-cell>
        <fo:block>
          $<xsl:value-of select="Price*Quantity"/>
        </fo:block>
      </fo:table-cell>
    </fo:table-row>
  </xsl:template>

  <xsl:template match="Tax|Shipping|Total">
    <fo:block font-size="16pt" font-family="serif"
              line-height="20pt">
      <xsl:value-of select="name()"/>: $<xsl:apply-templates/>
    </fo:block>
  </xsl:template>

  <!-- want to leave this one out of the output -->
  <xsl:template match="SKU"/>

</xsl:stylesheet>

The output vocabulary used in this stylesheet is XSL Formatting Objects. The actual document produced by transforming Example 1.5 with this style sheet is shown in Example 1.13 (with a few allowances for white space). This document would then be fed into an XSL-FO processor such as the Apache XML Project’s open source FOP to convert it to some other format, in this case PDF.

Example 1.13. An XSL-FO document for the clock order

<?xml version="1.0" encoding="utf-8"?>
<fo:root xmlns:fo="http://www.w3.org/1999/XSL/Format">
   <fo:layout-master-set>
      <fo:simple-page-master master-name="only">
         <fo:region-body margin-left="0.5in" margin-top="0.5in"/>
      </fo:simple-page-master>
   </fo:layout-master-set>
   <fo:page-sequence master-reference="only">
      <fo:flow flow-name="xsl-region-body">
         <fo:block font-size="16pt" font-family="serif" line-height="20pt">
       Ship to:
    </fo:block>
         <fo:block font-size="16pt" font-family="serif" margin-left="0.5in" line-height="20pt">Chez Fred</fo:block>
         <fo:table font-size="12pt" space-before="24pt" space-after="24pt">
            <fo:table-column column-width="2in"/>
            <fo:table-column column-width="1in"/>
            <fo:table-column column-width="1in"/>
            <fo:table-column column-width="1in"/>
            <fo:table-body>
               <fo:table-row font-weight="bold">
                  <fo:table-cell>
                     <fo:block>Product</fo:block>
                  </fo:table-cell>
                  <fo:table-cell>
                     <fo:block>Quantity</fo:block>
                  </fo:table-cell>
                  <fo:table-cell>
                     <fo:block>Unit Price</fo:block>
                  </fo:table-cell>
                  <fo:table-cell>
                     <fo:block>Subtotal</fo:block>
                  </fo:table-cell>
               </fo:table-row>
               <fo:table-row>
                  <fo:table-cell>
                     <fo:block>Birdsong Clock</fo:block>
                  </fo:table-cell>
                  <fo:table-cell>
                     <fo:block>12</fo:block>
                  </fo:table-cell>
                  <fo:table-cell>
                     <fo:block>$21.95</fo:block>
                  </fo:table-cell>
                  <fo:table-cell>
                     <fo:block>
          $263.4</fo:block>
                  </fo:table-cell>
               </fo:table-row>
               <fo:table-row>
                  <fo:table-cell>
                     <fo:block>Brass Ship's Bell</fo:block>
                  </fo:table-cell>
                  <fo:table-cell>
                     <fo:block>1</fo:block>
                  </fo:table-cell>
                  <fo:table-cell>
                     <fo:block>$144.95</fo:block>
                  </fo:table-cell>
                  <fo:table-cell>
                     <fo:block>
          $144.95</fo:block>
                  </fo:table-cell>
               </fo:table-row>
            </fo:table-body>
         </fo:table>
         <fo:block font-size="16pt" font-family="serif" line-height="20pt">Tax: $28.20</fo:block>
         <fo:block font-size="16pt" font-family="serif" line-height="20pt">Shipping: $8.95</fo:block>
         <fo:block font-size="16pt" font-family="serif" line-height="20pt">Total: $431.00</fo:block>
      </fo:flow>
   </fo:page-sequence>
</fo:root>

Figure 1.2. The PDF version of the clock order produced by XSL

Since this book focuses more on using XML for computer-to-computer communication rather than computer-to-human or human-to-human communication, I’m not going to spend very many pages on style sheets. Indeed, this is the last time you’re going to see them for the next fifteen chapters. However, you should at least be aware that XSLT is an extremely powerful approach for certain kinds of problems. Indeed once you’ve gained a little facility with XSLT, you’ll notice that it is often easier to write an XSLT style sheet to solve a problem than a classic procedural program in Java. Even when XSLT can’t do everything you need, you may find that it can solve a large part of your problem. XML systems are often designed as a chain of steps: first validation using a DTD or schema, then transformation using XSLT, and finally whatever custom processing you wish to apply using Java. Many apparently hard problems become much simpler and easier to tackle when broken down in this fashion. I’ll explore this further when we return to XSLT in Chapter 17.

Prev | Up | Next
Validity | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 28, 2001


Summary
Prev Chapter 1. XML for Data Next

Summary

XML is a standard textual markup language suitable for encoding almost any sort of data. It works very well for both unstructured narrative data written by people and for the record-oriented data common in computer applications. About the only thing it’s not really suitable for are bitmapped things such as photographs and recorded sound.

Logically an XML document is made up of nested elements. Each element has a name, a set of attributes and some content. The content can include plain text and/or other elements. The attributes are name value pairs associated with the element. Each document has a single topmost element called the root or document element. Since all non-root elements nest completely inside other elements, an XML document has a natural tree structure. Besides elements and text nodes, XML documents can also contain comments, processing instructions, an XML declaration, and a document type declaration.

Syntactically elements are delimited by tags that look like <Quantity>, </Quantity>, and <Quantity/>. <Quantity> is a start-tag that must be matched by the corresponding end-tag </Quantity>. The content of the Quantity element comes in-between these two tags. <Quantity/> is an empty-element tag that represents a Quantity element with no content. Attributes are indicated by name="value" pairs inside start-tags and empty-element tags. For example, <Quantity number="17"/> is an empty Quantity element that has a number attribute with the value 17.

Physically, an XML document is divided into storage units called entities. These entities can be files, database records, data structures in memory, or something else. The document entity contains the root element of the document. Parsed entities contain XML markup and that will be merged to form the entire document. Parsed entities are located via general entity references such as &anaconda; in the document entity or another parsed entity. Unparsed entities contain non-XML, possibly binary data that will be identified by ENTITY type attributes in the document.

Every XML document must be well-formed. Among other things this means, every start-tag must have a matching end-tag, every attribute value must be quoted, and only certain characters can be used in element names. If a document is not well-formed, it is not an XML document; and XML parsers will not accept it. Beyond well-formedness, documents that have a schema may be (but do not have to be) valid. A valid document adheres to all the constraints listed in the schema. Schema languages include Document Type Definitions (DTDs), the W3C XML Schema Language, and the XPath-based Schematron.

Since XML markup normally focuses on the structure and semantics of the contained information, before a document can be shown to a human reader, it must first be associated with a style sheet that tells the browser or other tool how to format the document for display to a person. The two most popular style languages are Cascading Style Sheets (CSS) and the Extensible Stylesheet Language (XSL). CSS is a non-XML declarative language for applying simple styles such as font-weight to elements of certain types. XSL is actually two separate XML applications, the XSL-FO page description language and the XSLT Turing-complete functional language. An XSLT style sheet is used to transform a source XML document into the XSL-FO vocabulary. However, XSLT can also be used to transform to other XML vocabularies such as XHTML.

Prev | Up | Next
Style sheets | Home | Chapter 2. XML Protocols: XML-RPC and SOAP
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 29, 2001


Chapter 2. XML Protocols: XML-RPC and SOAP
Prev Next

Chapter 2. XML Protocols: XML-RPC and SOAP

Table of Contents

XML as a Message Format
Envelopes
Data Representation
HTTP as a Transport Protocol
How HTTP Works
HTTP in Java
RSS
Customizing the Request
Query Strings
How POST Works
XML-RPC
Data Structures
Faults
Validating XML-RPC
SOAP
A SOAP Example
Posting SOAP documents
Faults
Encoding Styles
SOAP Headers
SOAP Limitations
Validating SOAP
Custom Protocols
Summary

Before we can explore the available APIs for processing XML documents with Java, we’re going to need a few good examples. For most of this book, my examples are going to focus on XML protocols. These are XML applications used for machine-to-machine exchange of information across the Internet over HTTP. In this chapter I’ll show you how such documents move from one machine to another, and how you can use Java to interpose yourself in the process. However, since this is not a book about network programming, I’m going to be careful to keep all the details of network transport separate from the generation and processing of XML documents. When you work with an XML document, you don’t care whether it came from a file, a network socket, a string, or something else.

Three such XML protocol applications are of particular interest. The first is a very straightforward application called RSS. RSS is used to exchange headlines and abstracts between different Web news sites. It is available in two versions, RSS 0.9.1, which is based on an early working draft of the Resource Description Framework (RDF), and RSS 1.0 which is based on the final W3C recommendation of RDF. Both variants are used on the Web today.

The second XML application I’ll investigate in some detail is XML-RPC. XML-RPC supports remote procedure calls across the Internet by passing method names and arguments embedded in an XML document over HTTP. The third example application is a more complex implementation of this idea called SOAP. Whereas XML-RPC uses only elements, SOAP adds attributes and namespaces as well. SOAP even lets the body of the message be an XML element from some other vocabulary, so it opens up a host of other interesting examples.

XML as a Message Format

One of the major uses of XML is for exchanging data between heterogenous systems. Given almost any collection of data, it’s straightforward to design some XML markup that fits it. Since XML is natively supported on essentially any platform of interest, you can send data encoded in such an XML application from point A to point B without worrying about whether point A and point B agree on how many bytes there are in a float, whether ints are big endian or little endian, whether strings are null delimited or use an initial length byte, or any of the myriad of other issues that arise when moving data between systems. As long as both ends of the connection agree on the XML application used, they can exchange information without worrying about what software produced the data. One side can use Perl and the other Java. One can use Windows and the other Unix. One can run on a mainframe and the other on a Mac. The document can be passed over HTTP, e-mail, NFS, BEEP, Jabber, or sneakernet. Everything except the XML document itself can be ignored.

The details of the XML markup used depend heavily on the information you’re exchanging. If you’re exchanging financial data, you might use the Open Financial Exchange (OFX). If you’re exchanging genetic codes, you might use the Gene Expression Markup Language (GEML). If you’re exchanging news articles in a syndication service, you might use NewsML. And if no standard XML application exists that fits your needs, you’ll probably invent your own; but whatever XML application you choose, there are certain features that will crop up again and again and that can benefit from standardization. These include the envelope used to pass the data and the representations of basic data types like integer and date.

Envelopes

When only two systems are involved, they only talk to each other, and they always send the same type of message, an envelope may not be needed. It’s enough for one system to send the other the message in the agreed upon XML format. However, when it’s actually many dozens, hundreds, or even thousands of different systems exchanging many different kinds of messages in many different ways, it’s useful to have some standards that are independent of the content of the message. This offers up some hope that when a message in an unrecognized format is received, it can still be processed in a reasonable fashion. For example, a system might receive a message ordering one thousand “Frodo Lives” buttons but not know how to handle that order. However, it may be able to read enough information from the envelope to route the request to the program that does know how to process the order.

In XML-RPC, essentially all the markup is the envelope and all the text content is the data inside the envelope. SOAP and RSS are a little more complex. For SOAP, the envelope is an XML document, and the data is too. In some ways RSS, especially RSS 1.0, is the most complex of all because it’s based on the relatively complex RDF syntax. RDF mixes the envelope and the data together so that you can’t point to any one element in the whole document and say “That’s the envelope,” or “That element is the data.” Instead, pieces of both the envelope and the data are intermingled throughout the complete document. In all three cases, however, it’s straightforward to extract the data from the envelope for further processing.

Data Representation

Another area that’s ripe for standardization is the proper representation of low-level data such as dates and numbers. Nobody really cares how many bytes there are in an int as long as there are enough to hold all the values they want to hold. Nobody really cares whether dates are written Day-Month-Year or Month-Day-Year as long as it’s easy to tell which is which. It doesn’t really matter how this information is passed, as long as there’s one standard way of doing it that everyone can agree on and process without excessive hassle.

In XML all data of any type must be passed as text. The proper textual representation of simple data types such as integer and date is trickier than most developers initially assume. For example, integers can be straightforwardly represented in the form 42, -76, +34562, 0, and so forth. The normal base-10 representation with optional plus or minus signs is fully adequate for most needs. However, consider the number 28562476535, the dollar value of Bill Gates’s Microsoft stock holdings alone as of July 24, 2002. This is a perfectly good integer, albeit a large one. However, it’s so large that trying to use it in many programs will lead to a crash or some other form of error.

Floating point numbers are even worse. Two different computers can look at an unambiguous string such as 65431987467.324345192 and interpret it as two different numbers. Dates cause problems even for humans. Is 07/04/01 the fourth of July, 2001? the fourth of July 1901? the seventh of April 2001? Some other date? These are all very really issues that cause real problems in systems today.

XML itself doesn’t standardize the text representation of data, but the W3C XML Schema Languages does. In particular, schemas define the 44 simple data types shown in Table 2.1. By assigning these types to particular elements, you can clearly state what a particular string means in a syntax everyone can understand. And if these types aren’t enough, the W3C XML Schema Language also lets you define new types that are combinations or restrictions of these basic types.

Table 2.1. Primitive Data Types defined in the W3C XML Schema Language

Type | Meaning
xsd:string | The schema equivalent of #PCDATA, any string of Unicode characters that may appear in an XML document
xsd:boolean | true, false, 1, 0
xsd:decimal | A decimal number such as 44.145629 or -0.32, with an arbitrary size and precision; similar to the java.math.BigDecimal class
xsd:float | The 4-byte IEEE-754 floating point number which best approximates the specified decimal string, same as Java’s float type
xsd:double | The 8-byte IEEE-754 floating point number which best approximates the specified decimal string, same as Java’s double type
xsd:integer | An integer of arbitrary size, similar to the java.math.BigInteger class
xsd:nonPositiveInteger | An integer less than or equal to zero
xsd:negativeInteger | An integer strictly less than zero
xsd:nonNegativeInteger | An integer greater than or equal to zero
xsd:long | An integer between -9223372036854775808 and +9223372036854775807 inclusive; equivalent to Java’s long primitive data type
xsd:int | An integer between -2147483648 and 2147483647 inclusive; equivalent to Java’s int primitive data type
xsd:short | An integer between -32768 and 32767 inclusive; equivalent to Java’s short primitive data type
xsd:byte | An integer between -128 and 127 inclusive; equivalent to Java’s byte primitive data type
xsd:unsignedLong | An integer between 0 and 18446744073709551615.
xsd:unsignedInt | An integer between 0 and 4294967295
xsd:unsignedShort | An integer between 0 and 65535
xsd:unsignedByte | An integer between 0 and 255
xsd:positiveInteger | An integer strictly greater than zero
xsd:duration | A length of time given in the ISO 8601 extended format: PnYnMnDTnHnMnS. The number of seconds can be a decimal or an integer. All the other values must be non-negative integers. For example, P1Y2M3DT4H5M6.7S is one year, two months, three days, four hours, five minutes, and 6.7 seconds.
xsd:dateTime | A particular moment of time on a particular day up to an arbitrary fraction of a second in the ISO 8601 format: CCYY-MM-DDThh:mm:ss. This can be suffixed with a Z to indicate coordinated universal time (UTC) or an offset from UTC. For example, Neil Armstrong set foot on the moon at 1969-07-20T21:28:00-06:00 by the clock in Houston mission control which is also known as 1969-07-21T02:28:00Z
xsd:time | A certain time of day on no particular day in the ISO 8601 format: hh:mm:ss.sss. A time zone specified as an offset from UTC is optional. For example, on most days I wake up around 07:00:00.000-05:00 and go to bed around 23:30:00.000-05:00.
xsd:date | A particular date in history given in ISO 8601 format: YYYYMMDD; e.g. 20010706 or 19690920.
xsd:gYearMonth | A certain month in a certain year; e.g. 2001-12 or 1999-03.
xsd:gYear | A year in the Gregorian calendar ranging from 0001 to 2001 to 9999, 10000, 10001 and beyond. Earlier dates can be represented as ‑0001, ‑0002, ‑0003, and so forth back to the Big Bang. There is no year zero, however.
xsd:gMonthDay | A specific day of a specific month in no particular year in the form ‑‑02-28. For example, Christmas comes on ‑‑12-25.
xsd:gDay | A particular day of no particular month in the form ‑‑‑01, ‑‑‑02, ‑‑‑03, through ‑‑‑31
xsd:gMonth | A particular month in no particular year in the form ‑‑01‑‑, ‑‑02‑‑, ‑‑03‑‑, through ‑‑12‑‑
xsd:hexBinary | Hexadecimal encoded binary data; each byte of the data is replaced by the two hexadecimal digits that represent its unsigned value
xsd:base64Binary | Base-64 encoded binary data
xsd:anyURI | An absolute or relative URL or a URN
xsd:QName | An optionally prefixed XML name such as SOAP-ENV:Body or Body. Unprefixed names must be in the default namespace.
xsd:NOTATION | The name of a notation declared in the current schema
xsd:normalizedString | A string in which carriage returns (\r), line feeds (\n) and tab (\t) characters should be treated the same as spaces
xsd:token | A string in which all runs of white space should be treated the same as a single space
xsd:language | An RFC 1766 language identifier such as en, fr-CA, or i-klingon
xsd:NMTOKEN | An XML name token
xsd:NMTOKENS | A white space separated list of XML name tokens
xsd:Name | An XML name
xsd:NCName | An XML name that does not contain any colons; that is, an unprefixed name
xsd:ID | An NCName which is unique among other things of ID type in the same document
xsd:IDREF | An NCName used as an ID somewhere in the document
xsd:IDREFS | A whitespace separated list of IDREFs
xsd:ENTITY | An NCName that has been declared as an unparsed entity in the document’s DTD
xsd:ENTITIES | A white space separated list of ENTITY names

Even without using schema validation or the full schema apparatus, you can use these types in your own documents. Simply attach an xsi:type attribute to any element identifying the type of that element’s content. The xsi prefix is mapped to the http://www.w3.org/2001/XMLSchema-instance namespace URI. Example 2.1 shows an XML document that uses these types to label different parts of an order document. Notice that some things that might naively be assumed to be numeric types are in fact strings.

Example 2.1. An XML document that labels elements with schema simple types

<?xml version="1.0" encoding="ISO-8859-1"?>
<Order xmlns:xsd="http://www.w3.org/2001/XMLSchema"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Customer id="c32" xsi:type="xsd:string">Chez Fred</Customer>
  <Product>
    <Name xsi:type="xsd:string">Birdsong Clock</Name>
    <SKU xsi:type="xsd:string">244</SKU>
    <Quantity xsi:type="xsd:positiveInteger">12</Quantity>
    <Price currency="USD"  xsi:type="xsd:decimal">21.95</Price>
    <ShipTo>
      <Street xsi:type="xsd:string">135 Airline Highway</Street>
      <City xsi:type="xsd:string">Narragansett</City>
      <State xsi:type="xsd:NMTOKEN">RI</State>
      <Zip xsi:type="xsd:string">02882</Zip>
    </ShipTo>
  </Product>
  <Product>
    <Name xsi:type="xsd:string">Brass Ship's Bell</Name>
    <SKU xsi:type="xsd:string">258</SKU>
    <Quantity xsi:type="xsd:positiveInteger">1</Quantity>
    <Price currency="USD" xsi:type="xsd:decimal">144.95</Price>
    <Discount xsi:type="xsd:decimal">.10</Discount>
    <ShipTo>
      <GiftRecipient xsi:type="xsd:string">
        Samuel Johnson
      </GiftRecipient>
     <Street xsi:type="xsd:string">271 Old Homestead Way</Street>
      <City xsi:type="xsd:string">Woonsocket</City>
      <State xsi:type="xsd:NMTOKEN">RI</State>
      <Zip xsi:type="xsd:string">02895</Zip>
    </ShipTo>
    <GiftMessage xsi:type="xsd:string">
      Happy Father's Day to a great Dad!

      Love,
      Sam and Beatrice
    </GiftMessage>
  </Product>
  <Subtotal currency='USD' xsi:type="xsd:decimal">
    393.85
  </Subtotal>
  <Tax rate="7.0"
       currency='USD' xsi:type="xsd:decimal">28.20</Tax>
  <Shipping method="USPS" currency='USD'
            xsi:type="xsd:decimal">8.95</Shipping>
  <Total currency='USD' xsi:type="xsd:decimal">431.00</Total>
</Order>

As well as explicit labeling, a document can use a schema to indicate the type. However, right now the APIs for such things aren’t finished so it’s best to explicitly label elements when the types are important.

XML-RPC only uses the int, boolean, decimal, dateTime, and base64 types as well as a string type that’s restricted to ASCII. It also does not allow the NaN, Inf, and -Inf values for double. It does not use xsi:type attributes, relying instead on predefined semantics for particular elements. SOAP allows all 44 types and does use xsi:type attributes to label elements.

Prev | Up | Next
Summary | Home | HTTP as a Transport Protocol
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 08, 2002


HTTP as a Transport Protocol
Prev Chapter 2. XML Protocols: XML-RPC and SOAP Next

HTTP as a Transport Protocol

XML is just a document format. Documents by themselves don’t do anything. They simply are. They contain information, but they neither read nor write that information. They tend to stay in one place, and do not move unless somebody or something moves them.

If two systems want to exchange messages in XML format, it is not hard for them to do so. Each message can be encoded as a complete XML document. The sender can transmit the document to the receiver using FTP, HTTP, NFS, named pipes, RPC, floppy disks, a null-modem cable running between two machines’ serial ports, modems communicating over telephone lines, or any other means of moving data between systems. It’s even acceptable for the sender to print out the XML document on paper, seal it in a stamped envelope, and drop it in the mail. After the Post Office delivers the mail, the recipient can scan it in. XML is completely transport-protocol neutral. As long as the document gets where it’s going without being corrupted along the way, XML neither knows nor cares how it got there.

Since XML doesn’t care how documents are moved from point A to point B, it’s sensible to pick the simplest broadly supported protocol you can; and that is HTTP, the Hypertext Transport Protocol used by all web browsers and servers. Using HTTP to transport XML has a number of advantages, among them:

  • HTTP is well supported by libraries in Java, Perl, C, and many other languages for both client and server programs. This takes a large burden off the shoulders of the programmer.

  • HTTP is platform independent. Windows PCs, Macs, Unix boxes, mainframes, and more are all happy speaking HTTP.

  • HTTP connections are normally allowed to pass through firewalls.

  • Since the HTTP protocol is text-based, you can use telnet to test out servers.

  • The HTTP header provides a convenient place to store out-of-band information such as the document size and encoding.

  • HTTP is very well understood in the developer community.

Thus it should be no surprise that one of the most popular ways to move XML documents between systems is by using HTTP. A server can send an XML document to a client just as easily as it can send an HTML document or a JPEG image. A little less well-known is that a client can easily send an XML document (or anything else) to a server using HTTP POST.

How HTTP Works

The simplest possible XML protocol merely pulls down an XML document from a known URL on a server. You don’t need to send anything more to the server than a request for a page. For instance, today’s Slashdot headlines are available in XML from http://www.slashdot.org/slashdot.xml. All you have to do to get the headlines is load that URL into a browser as shown in Figure 2.1.

Figure 2.1. Slashdot headlines in XML

Behind the scenes, here’s what’s going over the wire for that request. First the browser opens a socket to www.slashdot.org on port 80. Then it sends a request that looks something like this:

GET /slashdot.xml HTTP/1.1
Host: www.slashdot.org
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9.2)
Accept: text/xml, text/html;q=0.9, image/jpeg, */*;q=0.1
Accept-Language: en, fr;q=0.50
Accept-Encoding: gzip,deflate,compress,identity
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
Keep-Alive: 300
Connection: keep-alive

The server responds with its own HTTP header, a blank line, and the body of the requested document. The result looks like this:

HTTP/1.1 200 OK
Date: Fri, 20 Jul 2001 19:31:10 GMT
Server: Apache/1.3.12 (Unix) mod_perl/1.24
Last-Modified: Fri, 20 Jul 2001 18:34:06 GMT
ETag: "de47c-e5e-3b58799e"
Accept-Ranges: bytes
Content-Length: 3678
Connection: close
Content-Type: text/xml

<?xml version="1.0" encoding="ISO-8859-1"?><backslash
xmlns:backslash="http://slashdot.org/backslash.dtd">

  <story>
    <title>TheKompany's Shawn Gordon Responds In Full</title>
    <url>http://slashdot.org/article.pl?sid=01/07/20/1637220</url>
    <time>2001-07-20 18:30:28</time>
    <author>timothy</author>
    <department>what's-good-for-thekompany</department>
    <topic>linuxbiz</topic>
    <comments>3</comments>
    <section>interviews</section>
    <image>topiclinuxbiz.gif</image>
  </story>

  <story>
    <title>GNOME Usability Study Report</title>
    <url>http://slashdot.org/article.pl?sid=01/07/20/1752205</url>
    <time>2001-07-20 18:08:03</time>
    <author>michael</author>
    <department>press-ctrl-alt-shift-k-q-z-to-continue</department>
    <topic>gnome</topic>
    <comments>68</comments>
    <section>developers</section>
    <image>topicgnome.gif</image>
  </story>

  <story>
    <title>Mono Unimplementable?</title>
    <url>http://slashdot.org/article.pl?sid=01/07/20/1555256</url>
    <time>2001-07-20 17:14:39</time>
    <author>Hemos</author>
    <department>keeping-things-propietary</department>
    <topic>microsoft</topic>
    <comments>132</comments>
    <section>articles</section>
    <image>topicms.gif</image>
  </story>

</backslash>

In both directions the HTTP header is pure text. Lines in the header are delimited by carriage-return linefeed pairs; that is, \r\n. The body of the document is separated from the HTTP header by a blank line. The document may or may not be text. For instance, it could be a JPEG image or a gzipped HTML file.

Servers can also respond with a variety of error codes. For example, here’s the common 404 Not Found error:

HTTP/1.0 404 Not found
Server: Netscape-Enterprise/2.01
Date: Wed, 04 Jul 2001 20:35:17 GMT
Content-length: 207
Content-type: text/html

<TITLE>Not Found</TITLE><H1>Not Found</H1> The requested object does
not exist on this server. The link you followed is either outdated,
inaccurate, or the server has been instructed not to let you have it.

HTTP in Java

Of course, you don’t have to load these documents into a browser. Java lets you write programs that connect to and retrieve information from web sites with hardly any effort. Once you have a document in memory you can do whatever you want with it: search it, sort it, transform it, forward it to somebody else, clean up after the family dog with it, whatever you want. Future chapters are going to cover the details of all these operations (except cleaning up after the family dog; you’re on your own for that :-) ). But first I want to show you how to use Java to retrieve such information and dump it to the console. Later we’ll move from merely retrieving and printing a document to reading and understanding it.

Example 2.2 is a Java class that uses the java.net.URL class to load documents via HTTP (or any other supported protocol) from a server. It has four methods. getDocumentAsInputStream() connects to a server and returns the unread stream after stripping off the HTTP header. getDocumentAsString() actually reads the entire document, stores it in a string buffer, and then returns a string containing the document at the URL. Overloaded variants of each method allow you to pass in either the string form of the URL or a java.net.URL object, whichever is more convenient. The method that retrieves the document as an input stream would be used if you want to process the document as it arrives. The string version would be used if the document wasn’t too big and you wanted to make sure the entire document was available before working with it.

Example 2.2. URLGrabber

package com.macfaq.net;

import java.net.URL;
import java.io.IOException;
import java.net.MalformedURLException;
import java.io.InputStream;

public class URLGrabber {

  public static InputStream getDocumentAsInputStream(URL url)
    throws IOException {

    InputStream in = url.openStream();
    return in;

  }

  public static InputStream getDocumentAsInputStream(String url)
    throws MalformedURLException, IOException {

    URL u = new URL(url);
    return getDocumentAsInputStream(u);

  }

  public static String getDocumentAsString(URL url)
    throws IOException {

    StringBuffer result = new StringBuffer();
    InputStream in = url.openStream();
    int c;
    while ((c = in.read()) != -1) result.append((char) c);
    return result.toString();

  }

  public static String getDocumentAsString(String url)
    throws MalformedURLException, IOException {

    URL u = new URL(url);
    return getDocumentAsString(u);

  }

}

One method you might expect to see here I’ve deliberately left out. There is no getReaderFromURL(). Before you can convert the input stream into a reader, you need to figure out which encoding to use. Because XML documents normally carry their own information about encoding, this requires you to parse the first line or two of the XML document. URLGrabber could do that, but the details are rather complicated and doing this would make this class less generic than it currently is. (Right now it can handle any kind of document, not just XML documents.) It’s better to leave all the XML-specific details such as determining the character encoding to the XML parser.

Since URLGrabber is a generally useful class, I’ve placed it in the com.macfaq.net package. I will use this class again later in this book without further comment. To do so I will have to import com.macfaq.net.*, and I will have to make sure my source files are properly organized in my file system. The Java examples in this book are not all trivial, and cannot all fit in a single class. In general, I will divide programs into different classes offering small pieces of functionality like this one so that I can debug and explain them separately, as well as mix and match them in different programs.

Warning

Nothing in Java is as pointlessly confusing as the proper organization of .class and .java files in different packages on a file system. Learning how to use packages correctly is one of the major hurdles for novice Java programmers. It is also one of the obstacles a good IDE can really help you with. If you are having problems making these programs compile and work as described here because of the package structure, especially if you’re seeing error messages that involve “java.lang.NoClassDefFoundError”, please consult a good introductory reference on Java. For these details, I recommend [How the virtual machine locates classes, in Core Java 2, Volume 1, by Cay S. Horstmann and Gary Cornell, Sun Microsystems Press 2001, ISBN 0-13-089468-0, pp. 168-171] and [Managing Source and Class Files, in The Java Tutorial, 3rd Edition, by Mary Campione, Kathy Walrath, and Alison Hunt, Addison-Wesley, 2001, ISBN 0-201-70393-9, pp. 238-241]. (Neither of these is complete. In particular neither covers the crucial sourcepath option to the javac compiler, or shows you how to compile and run a program divided across multiple packages. I’m still looking for a better introductory reference on these topics. If you know of one, please drop an email to elharo@metalab.unc.edu. Thanks!) However, in this book I will assume that you have learned how to navigate the obstacles Java places in your CLASSPATH with a reasonable degree of facility.

Isolating the code to communicate with the network in the URLGrabber class will allow us to ignore it in the future. For the most part, the details of network transport are not relevant when processing XML. Many of the programs in this book that process XML will expect to receive an XML document as a stream or a string. They really don’t care where that stream or string comes from as long as it contains a well-formed XML document. In the remainder of this book when I need to load an XML document from a URL via GET, I will refer to this class without duplicating the code.

Another aspect of this program that I find sometimes confuses people: it does not have a main() method. It is not intended to be used directly by typing java URLGrabber at the command line. Rather this is a library class meant for other programs to use. Example 2.3 is a simple program designed just to test URLGrabber with a very basic command line user interface. Since it is not a generally useful class, I have not placed it anywhere in the com.macfaq packages. Instead it is simply a quick, one-off program so I can make sure that URLGrabber actually works and does what I want it to do.

Example 2.3. URLGrabberTest

import com.macfaq.net.URLGrabber;
import java.io.IOException;
import java.net.MalformedURLException;

public class URLGrabberTest {

  public static void main(String[] args) {

    for (int i = 0; i < args.length; i++) {
      try {
        String doc = URLGrabber.getDocumentAsString(args[i]);
        System.out.println(doc);
      }
      catch (MalformedURLException e) {
        System.err.println(args[i]
         + " cannot be interpreted as a URL.");
      }
      catch (IOException e) {
        System.err.println("Unexpected IOException: "
         + e.getMessage());
      }
    }

  }

}

Here’s a simple example of using URLGrabberTest to download the XML document from http://www.slashdot.org/slashdot.xml:

D:\books\XMLJAVA>java URLGrabberTest http://www.slashdot.org/slashdot.xml
<?xml version="1.0" encoding="ISO-8859-1"?><backslash
xmlns:backslash="http://slashdot.org/backslash.dtd">

  <story>
    <title>TheKompany's Shawn Gordon Responds In Full</title>
...

However, URLGrabberTest is really nothing more than toy example that fits nicely and neatly in the space available on a printed page. In 2002, serious programs either use GUIs or they don’t have an explicit user interface at all. Example 2.2 is a useful program. Example 2.3 is not.

There are many services that can use this simple approach to transmitting XML data across HTTP. For example, besides headlines this is a straightforward way to distribute the titles and show times of the movies playing at a particular theater, the viewing conditions at an observatory, the operational state of a machine in a factory, the daily sales at a retail store, the surf conditions at Ehukai Beach, and more. What makes this possible is that all users want pretty much the same document. The response does not need to be customized for each requestor.

Prev | Up | Next
Chapter 2. XML Protocols: XML-RPC and SOAP | Home | RSS
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 26, 2001


RSS
Prev Chapter 2. XML Protocols: XML-RPC and SOAP Next

RSS

This style of distributing XML content is particularly common on news sites that distribute their headlines. Many use a standard format called RSS to do this. Example 2.4 is one example RSS file from Luke Tymowski’s QubeQuorner on July 16, 2001. All you need to know is the URL where the constantly updated file is kept. You don’t need to send anything to the server except a request for this file.

Example 2.4. An RSS 0.91 document

<?xml version="1.0"?>
 <!DOCTYPE rss PUBLIC
          "-//Netscape Communications//DTD RSS 0.91//EN"
          "http://my.netscape.com/publish/formats/rss-0.91.dtd" >
  <rss version="0.91">
    <channel>
        <title>QubeQuorner</title>
        <link>http://q.seeto.com</link>
        <description>
          A weblog for both the Sun Qube and Linux/BSD users
        </description>
        <language>en</language>
        <managingEditor>
          luke@seeto.com (Luke Tymowski)
        </managingEditor>
        <webMaster>luke@seeto.com (Luke Tymowski)</webMaster>
        <item>
           <title>Switching from Eudora/Mac to Pine</title>
           <link>http://q.seeto.com/994897187</link>
           <description>inessential.com: Switching from
           Eudora/Mac to Pine "I recently switched from
           Eudora/Mac, which I used for many years, to Pine
           [on OS X]. Why the hell would I do a thing like that?
           I had a few reasons."</description>
           </item>
        <item>
           <title>Symmetric Cryptography in Perl</title>
           <link>http://q.seeto.com/994895485</link>
           <description>Perl.com: Symmetric Cryptography in Perl
           "Having purchased the $250 cookie recipe from
           Neiman-Marcus, Alice wants to send it to Bob, but keep
           it away from Eve, who snoops on everyone's network
           traffic from the cubicle down the hall. How can Perl
           help her?"</description>
           </item>
        </channel>
</rss>

This is a pretty standard example of an RSS document. The root element is rss which has a version attribute. Here the version is 0.91, the first version to achieve broad adoption, and still the version most prevalent on the Web today. The rss element contains a single channel element. The channel element begins with several child elements providing meta-information about the news site. Here title, link, description, language, managingEditor, and webMaster elements are used. Though not used here, the channel element can also contain language, rating, copyright, pubDate, lastBuildDate, skipHours, skipDays and several other elements. Finally the metadata is followed by multiple item elements, one for each story on the site. Most items contain a title for the story, a link giving the URL to that particular story, and a description offering a brief plain text abstract or summary of the story.

As you can see, RSS 0.91 is a very simple and straightforward format. It’s easy to understand and easy to process. It’s well-formed and valid XML. It is unfortunately not, however, legal RDF. Since the acronym RSS originally stood for “RDF Site Summary”, this is viewed by some as a concern. (Today it doesn’t really stand for anything.) The cause of the problem is that RSS achieved broad adoption before RDF was finished, and the syntax of RDF changed out from under RSS. Thus there is currently an effort to create a new version of RSS that is in fact legal RDF. Example 2.5 shows the same document from the Qube Quorner, but in RSS 1.0 instead of RSS 0.91.

Example 2.5. An RSS 1.0 document

<?xml version="1.0"?>

<!DOCTYPE rdf:RDF [
<!ENTITY % HTMLlat1 PUBLIC
 "-//W3C//ENTITIES Latin 1 for XHTML//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml-lat1.ent">
%HTMLlat1;
]>

<rdf:RDF
 xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
 xmlns="http://purl.org/rss/1.0/"
 xmlns:dc="http://purl.org/dc/elements/1.1/"
 xmlns:ag="http://purl.org/rss/1.0/modules/aggregation/"
 xmlns:admin="http://webns.net/mvcb/"
 xmlns:syn="http://purl.org/rss/1.0/modules/syndication/"
>

<channel rdf:about="http://q.seeto.com/">
<title>QubeQuorner</title>
<link>http://q.seeto.com/</link>
<description>a weblog for the Qube community</description>
<dc:rights>Copyright &#xA9; 2001 QubeQuorner.
           All rights reserved.</dc:rights>
<dc:publisher>Luke Tymowski (mailto:luke@seeto.com)</dc:publisher>
<dc:language>en-us</dc:language>
<dc:date>2001-06-09T10:30:21-05:00</dc:date>
<syn:updatePeriod>hourly</syn:updatePeriod>
<syn:updateFrequency>1</syn:updateFrequency>
<syn:updateBase>1970-01-01T00:00:00-05:00</syn:updateBase>
<admin:errorReportsTo rdf:resource="mailto:luke@seeto.com"/>
<items>
<rdf:Seq><rdf:li rdf:resource="http://q.seeto.com/994897187" />
<rdf:li rdf:resource="http://q.seeto.com/994895485" />
</rdf:Seq>
</items>
<textinput rdf:resource="http://q.seeto.com/" />
</channel>
  <item rdf:about="http://q.seeto.com/994897187">
    <title>Switching from Eudora/Mac to Pine</title>
    <link>http://q.seeto.com/994897187</link>
    <description>
      inessential.com: Switching from Eudora/Mac to Pine
      "I recently switched from Eudora/Mac, which I used for
      many years, to Pine [on OS X]. Why the hell would I do a
      thing like that? I had a few reasons."
    </description>
  </item>
  <item rdf:about="http://q.seeto.com/994895485">
    <title>Symmetric Cryptography in Perl</title>
    <link>http://q.seeto.com/994895485</link>
    <description>Perl.com: Symmetric Cryptography in Perl "Having
     purchased the $250 cookie recipe from Neiman-Marcus, Alice
     wants to send it to Bob, but keep it away from Eve, who
     snoops on everyone's network traffic from the cubicle down
     the hall. How can Perl help her?"</description>
  </item>

  <textinput rdf:about="http://q.seeto.com/">
    <title>Search</title>
    <description>
      Search for a phrase that appeared in QubeQuorner
    </description>
    <name>body</name>
    <link>http://q.seeto.com/search</link>
  </textinput>
</rdf:RDF>

The most obvious difference is that RSS 0.9.1 is monolithic whereas RSS 1.0 uses namespaces to mix together markup from RSS, RDF, the Dublin Core, and a couple more applications. It doesn’t try to do it all. This requires somewhat more effort from the developer to handle. However, it does offer longer-term extensibility and maintainability.

Not everyone agrees that RSS actually needs to be RDF, and consequently development has forked. There’s a parallel effort to develop an RSS 0.9.2 and 0.9.3 that adds a few useful features to RSS 0.9.1, but otherwise attempts to maintain the simplicity of RSS 0.9.1.

Prev | Up | Next
HTTP as a Transport Protocol | Home | Customizing the Request
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 22, 2002


Customizing the Request
Prev Chapter 2. XML Protocols: XML-RPC and SOAP Next

Customizing the Request

Services like Backslash and RSS work because not a lot of input is needed. The same document is requested repeatedly. This is acceptable for retrieving documents from a file system on a remote server, but sometimes a little more customization is required. The client not only wants to request an XML document. It wants to parameterize that document. For instance, it might want to ask for headlines that include certain keywords or articles posted between two dates. The standard HTTP means of accomplishing this is to place the request parameters in a query string that is either attached to the end of the URL or included as the body of the HTTP request.

Note

There are other ways to encode request parameters. For instance, Amazon let’s you query their database by putting the ISBN number in the path of the URL. However, this requires a relatively specialized HTTP server. The two methods I discuss here are the standard approaches supported by most servers.

Query Strings

A query string is just a list of name=value pairs, much like attributes in an XML document except that the values aren’t quoted and names can be repeated. In a query string, the fields are separated from each other by ampersands. For example, this is a query string with four fields: one named page with the value xml, one named mode with the value stock, one named symbol with the value IBM, and another named symbol with the value SUNW:

page=xml&mode=stock&symbol=IBM&symbol=SUNW

The characters allowed in URLs, including their query string parts, are the ASCII letters A to Z in both upper and lower case, the digits 0 through 9 and the punctuation characters - _, ., !, ~, *, ', (, and ). Except for these 71 characters, all other characters used in query string names and values must be x-www-form-urlencoded. ( :, /, &, ?, #, and = can also be used but only in specific roles within the URL. When used as parts of file names or query string values, they have to be encoded too.) In x-www-form-urlencoding, each character is first converted to UTF-8, and then each byte in the UTF-8 representation of that character is replaced by a percent sign and the two-hexadecimal digits that represent that byte.

For example, the dollar sign has Unicode code point 36 or 0x24 in hexadecimal. Its UTF-8 representation is the single byte with that value. Thus it is escaped in URLs as %24. The Greek letter ψ has Unicode code point 968, 3C8 in hexadecimal. It is encoded in UTF-8 as two bytes, 207 and 136. Thus, after converting these bytes to hexadecimal, ψ is encoded as %CF%88. As a special case, the space character can be replaced by the plus sign. Java includes a java.net.URLEncoder class that can encode any string in this format. Java 1.2 and later also includes a java.net.URLDecoder class that can decode a string in this format.

The simplest way to attach a query string to an HTTP request is to append it to a URL, separated from the rest of the URL by a question mark. For example, the NASDAQ makes quotes available in XML from their server at quotes.nasdaq.com. To request a quote for a stock, you ask the server quotes.nasdaq.com for the file quotes.dll; and you pass it a query string with three fields: page, mode, and symbol. Set the page field to xml, the mode field to stock, and the symbol field to the stock symbol for the company you’re interested in. For example, to get a quote for Red Hat, you would load the URL http://quotes.nasdaq.com/quote.dll?page=xml&mode=stock&symbol=RHAT into your browser as shown in Figure 2.2. If you’re connecting to the server manually, you would request the document /quote.dll?page=xml&mode=stock&symbol=RHAT like this:

GET /quote.dll?page=xml&mode=stock&symbol=RHAT HTTP/1.0
Host: quotes.nasdaq.com
Accept: text/xml, application/xml
Accept-Language: en, fr;q=0.50
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66

HTTP/1.1 200 OK
Server: Microsoft-IIS/5.0
Date: Mon, 16 Jul 2001 21:51:32 GMT
Content-Length: 2057
Content-Type: text/xml

<?xml version="1.0" ?>
<!DOCTYPE nasdaqamex-dot-com
  SYSTEM "http://nasdaq.com/reference/NasdaqDotCom.dtd">
<nasdaqamex-dot-com>
<equity-quote symbol="RHAT" ilx-symbol="RHAT"
    hyperfeed-symbol="RHAT" telesphere-symbol="RHAT">
<issue-name>Red Hat, Inc.</issue-name>
<market-status>C</market-status>
<market-center-code>Nasdaq-NM</market-center-code>
<issue-type-code>Common Stock</issue-type-code>
<todays-high-price>3.94</todays-high-price>
<todays-low-price>3.74</todays-low-price>
<fifty-two-wk-high-price>28.875</fifty-two-wk-high-price>
<fifty-two-wk-low-price>3.65</fifty-two-wk-low-price>
<last-sale-price>3.78</last-sale-price>
<net-change-price>-0.14</net-change-price>
<net-change-pct>-3.57%</net-change-pct>
<share-volume-qty>932800</share-volume-qty>
<previous-close-price>3.92</previous-close-price>
<best-bid-price>3.76</best-bid-price>
<best-ask-price>3.86</best-ask-price>
<best-bid-price session-type="AfterHours">3.76</best-bid-price>
<best-ask-price session-type="AfterHours">3.86</best-ask-price>
<current-pe-ratio>NE</current-pe-ratio>
<total-outstanding-shares-qty>
  168486000</total-outstanding-shares-qty>
<current-yield-pct>0</current-yield-pct>
<earnings-actual-eps-amt>-0.53</earnings-actual-eps-amt>
<cash-dividend-amt>0</cash-dividend-amt>
<cash-dividend-ex-date>19691231</cash-dividend-ex-date>
<sp500-beta-num>2.02</sp500-beta-num>
<trade-datetime>20010716 16:00:00</trade-datetime>
<issuer-address-line1-txt>
 2600 Meridian Parkway</issuer-address-line1-txt>
<issuer-city-state-zip-txt>
  Durham NC 27713 USA</issuer-city-state-zip-txt>
<issuer-phone-num> 919-547-0012</issuer-phone-num>
<issuer-web-site-url>http://www.redhat.com</issuer-web-site-url>
<issuer-logo-url>
http://a676.g.akamaitech.net/f/676/838/1h/nasdaq.com/logos/RHAT.GIF
</issuer-logo-url>
<trading-status>ACTIVE</trading-status>
<market-capitalization-amt>636877080</market-capitalization-amt>
<option-root-symbol symbol=""/>
<tick-code tick-type="last-sale"></tick-code>
<tick-code tick-type="best-bid"></tick-code>
<tick-code tick-type="best-ask"></tick-code>
</equity-quote>
</nasdaqamex-dot-com>

Figure 2.2. NASDAQ Stock Data Retrieved via a Query String

Most of the hard work here is on the server side. From a client perspective, you just appear to be requesting a file with a slightly different name. This approach to sending a query string to a server is sometimes known as CGI GET, though it’s not necessarily a CGI program that responds to the request. It could be a servlet, a PHP page, an ASP page, or something else.

When the response needs to be customized for different users but the information the client sends to the server isn’t too large, then don’t underestimate the power of CGI GET. It can be simpler to send a query string than a full XML document because you can take advantage of the many client and server side CGI libraries already available to you. The JDK includes standard classes for encoding and decoding data in the x-www-form-urlencoded format. However, limitations in much software does mean that query strings embedded in URLs are limited to about 200 characters. Furthermore, the data they can encode is pretty flat. A query string cannot represent complex, hierarchical structures very well. XML, of course, is ideal for such structures. To encode the request in XML as well as the response, we need to explore an alternative to the GET method called POST.

How POST Works

HTTP GET accounts for 90%+ of normal web browsing. The browser sends a small request for a document and the server sends an HTTP header followed by the requested document, or perhaps an error message. However, when you fill out a form and click the submit button, the process is a little different. In particular if the form uses the POST method, then the browser not only sends the request line and the HTTP header. It also sends the form data as the request body, separated from the header with a blank line. Customarily browsers send an x-www-form-urlencoded query string as the body of the request. A typical POST form submission looks something like this:

POST /cartmgr.cgi HTTP/1.1
Host: www.irs.gov
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9.2)
Accept: application/xml, text/html;q=0.9, image/png, */*;q=0.1
Accept-Language: en, fr;q=0.50
Accept-Encoding: gzip,deflate,compress,identity
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
Keep-Alive: 300
Connection: keep-alive
Content-type: application/x-www-form-urlencoded
Content-Length: 264

action=DISPLAY_CART&template=cartmgr.cart_display.html.txt
&error_template=default_error.html.txt
&Show+me+my+cart=Show+me+my+cart&action=DISPLAY_DOC
&CreditCard=1234567898769876&CardHolder=Elliotte+Harold
&expiresMonth=07&expiresYear=2003&type=Visa
&template=cartmgr.redirect.html.txt

Normally you have to send an x-www-form-urlencoded data query string in the body of a POST request because that’s what the server expects, and the CGI program on the server has to be prepared to read x-www-form-url-encoded query strings because that’s what browsers will send. However, if you control both the server and the client, then you aren’t limited to this format. You can send any kind of data you like in the HTTP request body, including a complete XML document! And indeed this is exactly what both XML-RPC and SOAP do.

Note

The java.net.URL class, query strings, x-www-form-urlencoding, the GET and POST methods, HTTP headers, HTTP response codes, and many other aspects of working with HTTP in Java are covered in much more detail in my book [Java Network Programming, O’Reilly & Associates, 2000, ISBN 0-13-089468-0]

Prev | Up | Next
RSS | Home | XML-RPC
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 26, 2001


XML-RPC
Prev Chapter 2. XML Protocols: XML-RPC and SOAP Next

XML-RPC

XML-RPC is an XML application designed to enable remote procedure calls (RPC) over the Internet. In Java lingo, a procedure call is just a method invocation. In some other languages like C it might be called a function call. However, it just means that some named chunk of code somewhere is invoked with a list of argument values of particular types. The procedure may or may not return a single value of a known type; i.e. in Java terms it may or may not return void. A remote procedure call is one in which the called procedure is not necessarily running on the same host as the calling procedure.

XML-RPC was hardly the first effort to invent a syntax for remote procedure calls. There have been numerous attempts previously including CORBA and Java’s own Remote Method Invocation (RMI). However, prior to XML-RPC none of these technologies lived up to their hype. The reason is they were too complex, too binary, too opaque, and too platform dependent.

XML-RPC is a clear case of the triumph of worse is better. It bit off the 90% of the problem that gave developers the features they actually needed. It ignored the 10% of the problem that caused 90% of the complexity in previous RPC systems. Features it ignores completely include garbage collection, object stubs and skeletons, callbacks, activation, object serialization, and more. What it provides is a simple, easy-to-comprehend means of sending a method name and a list of arguments from one system to another. And it turns out that this may be the only thing that’s needed in many, many actual systems.

At a very high level, the fundamental idea of XML-RPC is this: An XML document that contains a method name and some arguments is sent to a Web server using HTTP POST. The server invokes the method with the specified arguments. Then it wraps up the return value of the method in another XML document, and sends that back to the client.

For example, let’s suppose you want to invoke the getQuote() method in the program running at the URL http://stocks.cafeconleche.org/quotes.cgi. This method takes as an argument a string containing a stock symbol and returns a double containing the current price of the stock. That is, in Java parlance its signature looks like this:

public static double getQuote(String symbol);

Encoded as an XML-RPC document, the request looks quite different, but the same information is still present. Example 2.6 demonstrates.

Example 2.6. An XML-RPC request document

<?xml version="1.0"?>
<methodCall>
  <methodName>getQuote</methodName>
  <params>
    <param>
      <value><string>RHAT</string></value>
    </param>
  </params>
</methodCall>

The root element of an XML-RPC request document is methodCall. This has a methodName child element whose content is the ASCII name of the method to invoke. The methodCall element also has a params child element containing the arguments to pass to the method. Each argument is encoded as a param element that contains a value element. The value element contains a child element identifying the type of the param, and this child element contains the actual value passed, as an ASCII string. Here the type of the single argument is string. Other possibilities include int, boolean, double, dateTime.iso8601, and base64. i4 is allowed as an alias for int, but is not a different type.

The XML-RPC client (which is normally not a web browser) will post this document to the server as shown in Example 2.7 using the MIME media type text/xml. It must provide the correct content length so that the server knows when the client has finished sending.

Example 2.7. POSTing an XML-RPC request document

POST /quotes.cgi HTTP/1.0
Host: stocks.cafeconleche.org
Content-Type: text/xml
Content-length: 167

<?xml version="1.0"?>
<methodCall>
  <methodName>getQuote</methodName>
  <params>
    <param>
      <value><string>RHAT</string></value>
    </param>
  </params>
</methodCall>

The server then responds with an HTTP header and a response document as shown in Example 2.8. The HTTP header is the standard sort of HTTP header you’d see with any successful web request.

Example 2.8. An XML-RPC response

HTTP/1.0 200 OK
Date: Mon, 16 Jul 2001 20:12:37 GMT
Server: Apache/1.3.12 (Unix) mod_perl/1.24
Last-Modified: Mon, 16 Jul 2001 20:12:37 GMT
Content-Length: 140
Connection: close
Content-Type: text/xml

<?xml version="1.0"?>
<methodResponse>
  <params>
    <param>
      <value><double>4.12</double></value>
    </param>
  </params>
</methodResponse>

This is the basic format of all successful XML-RPC responses. A methodResponse root element contains a single params child element which contains a single param child element which contains a single value element. The value element contains a single value of a type indicated by its child element, double in this example. The only thing that can change is the type and content of the response data.

So far I’ve been talking as if the program on the server that receives and processes this request is a Java class with the appropriately named methods, but that isn’t necessarily so. The server program could be written in Perl, Python, C, C#, C++, AppleScript, Rexx, or any of dozens of other languages. Whatever language the program is written in, it may or may not have an actual procedure named “getQuote”. All that’s required is that the server which receives this document has some way of dispatching the request to an appropriate process. Sometimes the server may parse the XML and send the raw data to the processor. Other times, it may send the entire XML document. It might even transform the XML document into another form and send that. What happens on the server doesn’t really matter as long as it eventually sends back its answer as an XML document in the proper format. XML very neatly and almost completely decouples the implementation from the interface. In this respect, at least, XML-RPC is a huge improvement over competing technologies like RMI, DCOM, and CORBA, all of which make way too many assumptions about how the remote end of the connection is implemented.

Data Structures

The implicit XML-RPC model of how services are implemented is a lot closer to C than to Java. This is not to say you can’t use Java to write XML-RPC clients or servers — you most certainly can — just that an XML-RPC procedure call is more like a C function call than a Java method call and that XML-RPC data types are more like C’s data types than Java’s object types. Fortuitously, XML-RPC doesn’t have any concept of pointers (the root of all C’s evils); but it does have structs and arrays for handling combinations of values and lists of values. These constructs can nest. That is, a struct member can be another struct or an array; and an array element can be a struct or another array. This enables you to pass fairly complex data structures to remote procedures.

Arrays

An XML-RPC array is represented as an array element. The array element contains a single data element which contains zero or more value elements. These are the same value elements used in param elements. Thus each value element contains a type element such as int or string which contains the actual data. For example, here’s an array that contains four stock symbols, each of which is a string:

<array>
  <data>
    <value><string>RHAT</string></value>
    <value><string>SUNW</string></value>
    <value><string>ASKJ</string></value>
    <value><string>COVD</string></value>
  </data>
</array>

Unlike arrays in C or Java, the elements of an XML-RPC array do not have to share the same type. Here’s an array that contains a string and two doubles:

<array>
  <data>
    <value><string>RHAT</string></value>
    <value><double>4.12</double></value>
    <value><double>4.25</double></value>
  </data>
</array>

You can use an array element wherever you would use an int or string or other type element. For example, Example 2.9 is a request for four stock quotes:

Example 2.9. An XML-RPC request that passes an array as an argument

<?xml version="1.0"?>
<methodCall>
  <methodName>getQuote</methodName>
  <params>
    <param>
      <value>
        <array>
          <data>
            <value><string>RHAT</string></value>
            <value><string>SUNW</string></value>
            <value><string>ASKJ</string></value>
            <value><string>COVD</string></value>
          </data>
        </array>
      </value>
    </param>
  </params>
</methodCall>

The response might also contain an array with four prices, as shown in Example 2.10:

Example 2.10. An XML-RPC response document that returns an array

<?xml version="1.0"?>
<methodResponse>
  <params>
    <param>
      <value>
        <array>
          <data>
            <value><double>4.12</double></value>
            <value><double>13.68</double></value>
            <value><double>1.93</double></value>
            <value><double>0.78</double></value>
          </data>
        </array>
      </value>
    </param>
  </params>
</methodResponse>

In some sense this indicates an overloaded method since in one example the getQuote() method is taking a string and in another it’s taking an array of strings. However, this may not necessarily map to an overloaded method on the server. Indeed, it may not map to a method named getQuote() at all. This could all just be an illusion perpetrated by the server to provide an easy to understand client interface to a very differently organized system.

Structs

A struct is a collection of variables. For those readers who went straight to Java and never had the misfortune of struggling with C, a struct is a poor man’s class. It has fields but no methods, and all the fields are public.

An XML-RPC struct is represented by a struct element. Each member of the struct is represented by a member element. Each member element has a name child and a value child. For example, this struct lists a stock symbol and a limit price:

<struct>
  <member>
    <name>symbol</name>
    <value><string>RHAT</string></value>
  </member>
  <member>
    <name>limit</name>
    <value><double>2.25</double></value>
  </member>
</struct>

As with arrays, you can use a struct anywhere you’d use one of the simple type elements such as int or string. For example, Example 2.11 is an XML-RPC request that represents a limit order. In a limit order, three values are required: the stock to buy, the price you’re willing to buy at, and the expiration date of the order.

Example 2.11. An XML-RPC Request that passes a struct as an argument

<?xml version="1.0"?>
<methodCall>
  <methodName>bid</methodName>
  <params>
    <param>
      <value>
        <struct>
          <member>
            <name>symbol</name>
            <value><string>RHAT</string></value>
          </member>
          <member>
            <name>limit</name>
            <value><double>2.25</double></value>
          </member>
          <member>
            <name>expires</name>
            <value><dateTime.iso8601>20020709T20:00:00</dateTime.iso8601></value>
          </member>
        </struct>
      </value>
    </param>
  </params>
</methodCall>

Responses can also contain structs. You’ll see an example of this in the next section when we talk about faults.

Faults

It’s not at all uncommon for a procedure call to fail. In Java this causes an exception. In XML-RPC this causes a fault. For example, getQuote() might fail if the client passes in a nonexistent stock symbol. A fault response is almost exactly like a successful response except that a fault element replaces the params element. The value of the fault is always a struct containing two members: faultCode, an int, and faultString, a string. Example 2.12 demonstrates a fault for an unknown stock symbol.

Example 2.12. An XML-RPC fault

<?xml version="1.0"?>
<methodResponse>
  <fault>
    <value>
      <struct>
        <member>
          <name>faultCode</name>
          <value><int>23</int></value>
        </member>
        <member>
          <name>faultString</name>
          <value><string>Unknown stock symbol ABCD</string></value>
        </member>
      </struct>
    </value>
  </fault>
</methodResponse>

When the server faults, the HTTP request still succeeds. The server still uses the 200 OK response code in the HTTP header. Other HTTP error codes like 404 Not Found or 500 Internal Server Error are returned only if something goes wrong with the request at the HTTP level, not in the XML-RPC invocation.

Validating XML-RPC

There’s no official DTD or schema for XML-RPC. Nonetheless, it’s straightforward to write one or both. Indeed such a DTD may be a more easily understandable description of what is and isn’t allowed than the prose specification.

A DTD for XML-RPC

Example 2.13 is a simple DTD for XML-RPC. It states that a methodCall contains one methodName and one params in that order, that a methodResponse contains one params or one fault, that a value element can contain an i4, int, string, datetime.iso8601, double, base64, struct, or array, and so forth.

Example 2.13. A DTD for XML-RPC

<!ELEMENT methodCall (methodName, params)>
<!ELEMENT methodName (#PCDATA)>
<!ELEMENT params     (param*)>
<!ELEMENT param      (value)>
<!ELEMENT value
   (i4|int|string|dateTime.iso8601|double|base64|struct|array)>

<!ELEMENT i4               (#PCDATA)>
<!ELEMENT int              (#PCDATA)>
<!ELEMENT string           (#PCDATA)>
<!ELEMENT dateTime.iso8601 (#PCDATA)>
<!ELEMENT double           (#PCDATA)>
<!ELEMENT base64           (#PCDATA)>

<!ELEMENT array            (data)>
<!ELEMENT data             (value*)>
<!ELEMENT struct           (member+)>
<!ELEMENT member           (name, value)>
<!ELEMENT name             (#PCDATA)>

<!ELEMENT methodResponse   (params | fault)>
<!ELEMENT fault            (value)>

There are also many things this DTD does not say. For example, it does not say that the value inside a fault must be a struct or that each i4 element contains an integer between -2,147,483,648 and 2,147,483,647. DTDs cannot make statements such as these. Schemas, however, can.

This DTD is informative, not normative. There is no official DTD for XML-RPC. I just made this one up after reading the XML-RPC specification. You can include a document type declaration in your XML-RPC documents, but it’s much more common to leave it out as is the case for all the examples in this chapter. Furthermore many parsers used in practice for reading XML-RPC documents are not validating and will not consider the contents of any DTD.

A Schema for XML-RPC

Example 2.14 is a medium complex schema for XML-RPC. It says everything the DTD says and then some. All elements are strictly typed. In the case of string and boolean, the XML-RPC types don’t quite match the schema types so new, more restricted types were derived from the standard base types. In addition the complex types are context dependent. For instance, structs normally contain one or more members, but the struct inside a fault always contains exactly two members, one of which has the name faultCode and the other of which has the name faultString.

Example 2.14. A Schema for XML-RPC

<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  <!-- The only two possible root elements are methodResponse and
       methodCall so these are the only two I use a top-level
       declaration for. -->

  <xsd:element name="methodCall">
    <xsd:complexType>
      <xsd:all>
        <xsd:element name="methodName">
          <xsd:simpleType>
            <xsd:restriction base="ASCIIString">
              <xsd:pattern value="([A-Za-z0-9]|/|\.|:|_)*" />
            </xsd:restriction>
          </xsd:simpleType>
        </xsd:element>
        <xsd:element name="params" minOccurs="0" maxOccurs="1">
          <xsd:complexType>
            <xsd:sequence>
              <xsd:element name="param"  type="ParamType"
                           minOccurs="0" maxOccurs="unbounded"/>
            </xsd:sequence>
          </xsd:complexType>
         </xsd:element>
      </xsd:all>
    </xsd:complexType>
  </xsd:element>

  <xsd:element name="methodResponse">
    <xsd:complexType>
      <xsd:choice>
        <xsd:element name="params">
          <xsd:complexType>
            <xsd:sequence>
              <xsd:element name="param" type="ParamType"/>
            </xsd:sequence>
          </xsd:complexType>
        </xsd:element>
        <xsd:element name="fault">
          <!-- What can appear inside a fault is very restricted -->
          <xsd:complexType>
            <xsd:sequence>
              <xsd:element name="value">
                <xsd:complexType>
                  <xsd:sequence>
                    <xsd:element name="struct">
                      <xsd:complexType>
                        <xsd:sequence>
                          <xsd:element name="member"
                                       type="MemberType">
                          </xsd:element>
                          <xsd:element name="member"
                                       type="MemberType">
                          </xsd:element>
                        </xsd:sequence>
                      </xsd:complexType>
                    </xsd:element>
                  </xsd:sequence>
                </xsd:complexType>
              </xsd:element>
            </xsd:sequence>
          </xsd:complexType>
         </xsd:element>
      </xsd:choice>
    </xsd:complexType>
  </xsd:element>

  <xsd:complexType name="ParamType">
    <xsd:sequence>
      <xsd:element name="value" type="ValueType"/>
    </xsd:sequence>
  </xsd:complexType>

  <xsd:complexType name="ValueType" mixed="true">
    <!-- I need to figure out how to say that this
         is either a simple xsd:string type or that
         it contains one of these elements; but that otherwise
         it does not have mixed content -->
    <xsd:choice>
      <xsd:element name="i4"            type="xsd:int"/>
      <xsd:element name="int"           type="xsd:int"/>
      <xsd:element name="string"        type="ASCIIString"/>
      <xsd:element name="double"        type="xsd:decimal"/>
      <xsd:element name="Base64"        type="xsd:base64Binary"/>
      <xsd:element name="boolean"       type="NumericBoolean"/>
      <xsd:element name="dateTime.iso8601" type="xsd:dateTime"/>
      <xsd:element name="array"         type="ArrayType"/>
      <xsd:element name="struct"        type="StructType"/>
    </xsd:choice>
  </xsd:complexType>

  <xsd:complexType name="StructType">
    <xsd:sequence>
      <xsd:element name="member" type="MemberType"
                   maxOccurs="unbounded"/>
    </xsd:sequence>
  </xsd:complexType>

  <xsd:complexType name="MemberType">
    <xsd:sequence>
      <xsd:element name="name"  type="xsd:string" />
      <xsd:element name="value" type="ValueType"/>
    </xsd:sequence>
  </xsd:complexType>

  <xsd:complexType name="ArrayType">
    <xsd:sequence>
      <xsd:element name="data">
        <xsd:complexType>
          <xsd:sequence>
            <xsd:element name="value"  type="ValueType"
                         minOccurs="0" maxOccurs="unbounded"/>
          </xsd:sequence>
        </xsd:complexType>
      </xsd:element>
    </xsd:sequence>
  </xsd:complexType>

  <xsd:simpleType name="ASCIIString">
    <xsd:restriction base="xsd:string">
      <xsd:pattern value="([ -~]|\n|\r|\t)*" />
    </xsd:restriction>
  </xsd:simpleType>

  <xsd:simpleType name="NumericBoolean">
    <xsd:restriction base="xsd:boolean">
      <xsd:pattern value="0|1" />
    </xsd:restriction>
  </xsd:simpleType>

</xsd:schema>

This schema is informative, not normative. There is no official schema for XML-RPC. I just wrote this one up after reading the XML-RPC specification. You should not include xsi:noNamespaceSchemaLocation attributes in your XML-RPC documents, but you might be able to use some other extra-document means of attaching the schema. For instance, the Xerces‑J parser lets you set the http://apache.org/xml/properties/schema/external-noNamespaceSchemaLocation property to the location of the schema. However, very few XML-RPC services would be likely to support this. If you use a schema with XML-RPC, it should be used purely for validation, not for attaching default attributes to elements or anything else that might affect the document’s content.

This example does effectively demonstrate the relative power of DTDs and schemas. The schema identifies all possible, legal XML-RPC documents. Almost anything that satisfies this schema is a legal XML-RPC document, and anything that does not satisfy the schema is not a legal XML-RPC document. [1] The DTD only gets half that far. All documents that do not satisfy the DTD are not legal XML-RPC documents. However, not all documents that do satisfy the DTD are legal XML-RPC documents. Things the schema says that the DTD does not include:

  • The only two legal root elements are methodCall and methodResponse

  • A params element that’s a child of a methodCall can have any number of param child elements, but a params element that’s a child of a methodResponse element must have exactly one param child element (i.e. context dependent content models)

  • i4 and int elements must contain an integer between -2,147,483,648 and 2,147,483,647.

  • String values and method names can only use ASCII characters.

  • The value of a fault must be a struct with exactly two members.

There are other things the schema says that the DTD doesn’t, but this gives you the idea. Schemas are both more descriptive and proscriptive than DTDs.

[1] The three XML-RPC requirements that can't be specified in the W3C XML Schema Language are:

  • One of the two members in a fault struct must have the name faultCode and an int value and the other must have the name faultString and a string value.

  • A value element can contain either an ASCII string or a type element such as int, but not a type element and an ASCII string.

Prev | Up | Next
Customizing the Request | Home | SOAP
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified August 22, 2002


SOAP
Prev Chapter 2. XML Protocols: XML-RPC and SOAP Next

SOAP

XML-RPC was in large part invented by a single person who really didn’t know a lot about XML. Consequently he made many very questionable choices; and since XML-RPC did not go through any sort of standardization process, there was nobody to fix his mistakes. For instance, in XML-RPC the string type is defined as an “ASCII string”. Now frankly, this is just plain dumb, as well as not a little ethnocentric. XML documents are Unicode, not ASCII. Modern programming languages like Java can handle Unicode without any trouble. Indeed a language that can’t process Unicode really isn’t suitable for processing XML. There is no good reason to limit XML-RPC strings to ASCII. I certainly wouldn’t say you have to use non-ASCII characters in your XML-RPC documents, but if you want to they should certainly be allowed. However, the inventor of XML-RPC also happened to be the vendor of an ASCII-limited database, so he inserted the ASCII-only constraint into XML-RPC so he wouldn’t have to upgrade his database to support Unicode.

There are a lot of other issues like that with XML-RPC, some equally obvious, some more subtle. Nonetheless, clearly XML-RPC was a good idea in principle if not execution. Consequently work began on a more serious effort to enable remote procedure calls by passing XML documents over HTTP. This effort is known as the Simple Object Access Protocol, or just SOAP. Whereas XML-RPC was a quick hack by one developer, SOAP has been developed by a committee of XML experts from various companies including IBM and Microsoft.

You’ve undoubtedly heard the old saw about a camel being a horse designed by committee. However, the fact is a camel is actually superbly adapted to its environment. SOAP is a much more robust protocol than XML-RPC. It is much better designed from an XML standpoint as well. It takes advantage of numerous features of XML such as attributes, Unicode, and namespaces that XML-RPC either ignores or actively opposes. XML-RPC is adequate for simple tasks. However, if you get serious with it you rapidly hit a wall. SOAP can take you a lot farther. Although there are some basic services available using XML-RPC, the future clearly lies with SOAP.

The biggest conceptual difference between SOAP and XML-RPC is that XML-RPC exchanges a limited number of parameters of six fixed types, plus structs and arrays. However, SOAP allows you to send the server arbitrary XML elements. This is a much more flexible approach.

A SOAP Example

Let’s investigate how the stock quote example would likely be implemented in SOAP. Encoded as a SOAP document, the request document looks quite different, but the same information is present as demonstrated in Example 2.15.

Example 2.15. A SOAP document requesting the current stock price of Red Hat

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" >
  <SOAP-ENV:Body>
    <getQuote xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <symbol>RHAT</symbol>
    </getQuote>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The most obvious difference between this document and the XML-RPC equivalent in Example 2.6 is the use of namespaces. Namespaces allow the method request to be an arbitrary XML element. This goes way beyond passing just a method name and some argument values. SOAP permits much more complex XML messages than does XML-RPC.

The server’s response is equally flexible. Example 2.16 demonstrates:

Example 2.16. A SOAP Response

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP-ENV:Body>
    <Quote xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <Price>4.12</Price>
    </Quote>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

These two examples are minimal SOAP documents. The root element of every SOAP document is Envelope which must be in the http://schemas.xmlsoap.org/soap/envelope/ namespace, at least in SOAP 1.1. (The URL will change in SOAP 1.2.) Normally a prefix is used, and as always you can pick any prefix as long as the URI stays the same. In this chapter, I always assume that the prefix SOAP-ENV is mapped to that namespace URI. (This is the prefix the SOAP 1.1 specification uses.)

Each SOAP-ENV:Envelope element contains exactly one SOAP-ENV:Body element. The content of this element is one or more XML elements specific to the service. These examples use Quote, getQuote, and Price elements in the http://namespaces.cafeconleche.org/xmljava/ch2/ namespace. Other services will use other elements from other namespaces. It’s also permissible to use elements from no namespace at all, though using namespaces is highly recommended.

Posting SOAP documents

Currently, most SOAP messages are passed over HTTP using POST, just like XML-RPC messages. Other transport protocols such as SMTP, BEEP, and Jabber can be supported as well. However, there are a couple of crucial differences in the HTTP headers used for SOAP:

  • The HTTP request header must contain a SOAPAction field.

  • If the SOAP request fails, the server should return an HTTP 500 Internal Server Error rather than 200 OK.

The SOAPAction field alerts web servers and firewalls that they’re dealing with a SOAP message. This allows firewalls to more easily filter SOAP requests without looking at the request body. The value of the SOAPAction field is a double quoted URI that somehow indicates the intent of the message. For instance, if Example 2.15 were POSTed to a servlet running on www.ibiblio.org under the control of the user elharo, then you might use the SOAPAction http://www.ibiblio.org/#elharo to indicate to the server and firewall who was responsible for processing this request. This is shown in Example 2.17.

Example 2.17. A SOAP document requesting the current stock price of Red Hat

POST /xml/cgi-bin/SOAPHandler HTTP/1.1
Content-Type: text/xml; charset="utf-8"
Content-Length: 267
SOAPAction: "http://www.ibiblio.org/#elharo"

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" >
  <SOAP-ENV:Body>
    <getQuote xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <symbol>RHAT</symbol>
    </getQuote>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Conceptually, SOAPAction URIs are very similar to namespace URIs because they aren’t meant to be resolved. They’re just a convenient way of assigning unique identifiers to certain classes of SOAP messages. There’s no particular standard for how they’re chosen. You might use the full absolute URL that receives the SOAP request or you might use some previously agreed upon URI. You can even use nothing at all. But the SOAPAction header must be present in order for the request to be identified as a SOAP request.

The server will normally send the response back to the client over the same socket the client used to send the request and then close the connection. Like any other HTTP response, a SOAP response begins with an HTTP return code, message, and header. Assuming the request was successful, then the response code is 200 OK. Unlike the request, the response does not use any special header fields beyond those used by regular web browsers and servers. Example 2.18 demonstrates:

Example 2.18. A SOAP document returning the current stock price of Red Hat

HTTP/1.0 200 OK
Content-Type: text/xml; charset="utf-8"
Content-Length: 260

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP-ENV:Body>
    <Quote xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <Price>4.12</Price>
    </Quote>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Faults

It’s a fact of life that requests fail. They may fail for reasons beyond control of the SOAP provider. For instance, you may launch your SOAP request into the ether just before the phone company severs the wire connecting you to the Internet while hooking up your neighbor’s new DSL line. That sort of failure would make it itself manifest at a lower layer, below XML and SOAP, probably as a SocketException if you’re working in Java.

It’s also possible that your request will successfully arrive at the server, only to find that the server doesn’t recognize the URL you’re posting to. In fact, the server might not even be configured to support SOAP requests. This sort of error would not throw an exception, but would return a 404 Not Found page rather than the expected SOAP response. Your code should be prepared to handle such events.

Finally, it’s also possible that the SOAP responder itself is able to be reached and correctly invoked, but that it cannot process the request. This may be because the request contained bad data (e.g. a symbol for a stock that doesn’t exist) or simply because the server code is buggy and encountered a problem. In these cases the SOAP server itself is responsible for producing the correct error message. This error message is a SOAP response with a SOAP-ENV:Envelope and a SOAP-ENV:Body, just like a normal response. However, the SOAP-ENV:Body must contain exactly one SOAP_ENV:Fault element and may not contain anything else.

The SOAP_ENV:Fault element contains up to four child elements:

faultcode

A faultcode element contains a qualified name such as SOAP-ENV:VersionMismatch that identifies the fault.

faultstring

A faultstring element contains a plain text message for human readers describing the fault.

faultactor

The optional faultactor element contains a URI identifying the node that generated the fault. It’s used when a SOAP request is passed through a chain of handlers. This element is optional.

faultdetail

A faultdetail element is used when the fault is specifically related to the body of the request (e.g. the stock symbol was not recognized) as opposed to the envelope. It contains child elements describing the fault. This element is present if and only if the fault was related to the SOAP body as opposed to the SOAP header.

Caution

These four elements are not namespace qualified, which is a little surprising. They are not in the http://schemas.xmlsoap.org/soap/envelope/ namespace. They are not in some other namespace. They are in no namespace at all.

SOAP defines four specific fault codes in the http://schemas.xmlsoap.org/soap/envelope/ namespace to indicate common conditions in a generic way. These are:

SOAP-ENV:VersionMismatch

The namespace of the SOAP-ENV element indicates that this message is intended for a server implementing a different version of the SOAP protocol; e.g. a SOAP 1.2 message has been sent to a SOAP 1.1 server.

SOAP-ENV:MustUnderstand

There’s something in the header that the message says the server has to understand before acting, but the server does not recognize. (I’ll talk about this soon in the section on SOAP headers.)

SOAP-ENV:Client

The client sent a message that is somehow defective. Perhaps it omitted a key piece of information the server needs. For instance, the getQuote message was sent and understood, but the getQuote element did not have a symbol child. The client is to blame for the problem.

SOAP-ENV:Server

The client sent a correctly formed message with all the necessary information, but some error prevented the server from processing it. For example, the server may have needed to connect to a remote database to retrieve some information, and the database server had crashed. The server is to blame for the problem.

Example 2.19 is a fault that might be returned in response to a request for the non-existent stock ABCD. The faultcode element is set to SOAP-ENV:Client to indicate that the client's request was incorrect. The faultstring element just contains a brief string of unmarked up text that can be used to more fully describe the problem to a human reader. The faultdetail content includes elements in the same namespace as the successful response, http://namespaces.cafeconleche.org/xmljava/ch2/. Since this request was processed by a single node, no faultactor element is necessary.

Example 2.19. A SOAP fault response

HTTP/1.0 500 Internal Server Error
Content-Type: text/xml; charset="utf-8"
Content-Length: 498

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:stock="http://namespaces.cafeconleche.org/xmljava/ch2/">
  <SOAP-ENV:Body>
    <SOAP-ENV:Fault>
      <faultcode>SOAP-ENV:Client</faultcode>
      <faultstring>
        There is no stock with the symbol ABCD.
      </faultstring>
      <faultdetail>
        <stock:InvalidSymbol>ABCD</stock:InvalidSymbol>
      </faultdetail>
    </SOAP-ENV:Fault>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Encoding Styles

The information encoded in the SOAP documents you’ve seen so far has all been nothing more than Unicode text strings. When you want to encode other types such as integers, arrays, and objects, you need to specify how the characters that make up the XML document should be deserialized into the local platform’s understanding of those types. For example, if a Java program encounters the element <Price>4.12</Price> should it convert this into a double? a float? a java.lang.String? a java.math.BigDecimal? a custom Price class? something else?

Any element in a SOAP document can have a SOAP-ENV:encodingStyle attribute whose value is a URI pointing to some kind of schema that says what types are assigned to which elements. The most common language to use for this schema is the W3C XML Schema Language. However, other schema languages such as RELAX NG are allowed too.

Example 2.20 uses the SOAP-ENV:encodingStyle attribute on the getQuote element to point to a schema at the relative URL trading.xsd. This schema defines the symbol element as having the custom type StockSymbol, and is shown in Example 2.21. This schema is just used for assigning types. It is not used for validation, though with a little extra work it could be.

Example 2.20. A SOAP document that specifies the encoding style

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP-ENV:Body>
    <getQuote xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/"
              SOAP-ENV:encodingStyle="trading.xsd">
      <symbol>RHAT</symbol>
    </getQuote>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Example 2.21. A schema that assigns type to elements in the http://namespaces.cafeconleche.org/xmljava/ch2/ namespace

<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://namespaces.cafeconleche.org/xmljava/ch2/"
xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/"
elementFormDefault="qualified">

  <xsd:element name="getQuote">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="symbol" type="StockSymbol"
                     maxOccurs="unbounded"/>
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>

  <xsd:simpleType name="StockSymbol">
    <xsd:restriction base="xsd:string">
      <!-- two to six upper case letters -->
      <xsd:pattern value="[A-Z][A-Z][A-Z]?[A-Z]?[A-Z]?[A-Z]?"/>
    </xsd:restriction>
  </xsd:simpleType>

</xsd:schema>

The SOAP-ENV:encodingStyle attribute can be placed on any element in the document. It applies to that element and its descendants, and overrides the schemas declared on any ancestor. It is commonly placed on the root SOAP-ENV:Envelope element.

SOAP singles out one encoding style for special treatment. If the SOAP-ENV:encodingStyle attribute has the value http://schemas.xmlsoap.org/soap/encoding/, then a predefined set of types are available that includes one element for each simple type defined in the W3C XML Schema Language and listed in Table 2.1. For instance, assuming the SOAP-ENC prefix is bound to the http://schemas.xmlsoap.org/soap/encoding/ URI (Note that this is not the same as the namespace URI or prefix for the SOAP envelope) then an int can be placed in a SOAP-ENC:int element like this

<SOAP-ENC:int>12</SOAP-ENC:int>

The complete list of types and their normal Java semantics is given in Table 2.2, though this really just mirrors Table 2.1. In many cases, Java does not have a type that exactly matches one of the derived types. Thus it uses the broader base class. For example, Java does not have an unsigned integer type but all values of type xsd:unsignedInt can fit into a Java long. Java does not have a PositiveInteger class, but all xsd:positiveIntegers can be represented by a java.math.BigInteger. In some cases the mapping is obvious. In others, different programs may use different Java types and objects to deserialize the same values. For instance, an xsd:int is exactly a Java int, and an xsd:double is as close to a Java double as it’s possible for a base-10 string to be. However, an xsd:anyURI, could reasonably be converted to a java.net.URL, a java.lang.String, or some custom URI class.

Table 2.2. Simple Value Elements defined in SOAP

SOAP type | Java type
SOAP-ENC:string | java.lang.String
SOAP-ENC:boolean | boolean
SOAP-ENC:decimal | java.math.BigDecimal
SOAP-ENC:float | float
SOAP-ENC:double | double
SOAP-ENC:integer | java.math.BigInteger
SOAP-ENC:positiveInteger | java.math.BigDecimal
SOAP-ENC:nonPositiveInteger | java.math.BigInteger
SOAP-ENC:negativeInteger | java.math.BigInteger
SOAP-ENC:nonNegativeInteger | java.math.BigInteger
SOAP-ENC:long | long
SOAP-ENC:int | int
SOAP-ENC:short | short
SOAP-ENC:byte | byte
SOAP-ENC:unsignedLong | double or java.math.BigInteger
SOAP-ENC:unsignedInt | long
SOAP-ENC:unsignedShort | int
SOAP-ENC:unsignedByte | int
SOAP-ENC:duration | custom class
SOAP-ENC:dateTime | java.util.Date
SOAP-ENC:time | java.sql.Time
SOAP-ENC:date | java.sql.Date
SOAP-ENC:gYearMonth | custom class
SOAP-ENC:gYear | custom class, int, or java.math.BigInteger
SOAP-ENC:gMonthDay | custom class
SOAP-ENC:gDay | custom class or int
SOAP-ENC:gMonth | custom class or int
SOAP-ENC:hexBinary | byte[]
SOAP-ENC:base64Binary | byte[]
SOAP-ENC:anyURI | java.net.URL, java.lang.String, or a custom class
SOAP-ENC:QName | java.lang.String or a custom class
SOAP-ENC:NOTATION | org.w3c.dom.Notation
SOAP-ENC:normalizedString | java.lang.String
SOAP-ENC:token | java.lang.String
SOAP-ENC:language | java.lang.String or a custom class
SOAP-ENC:NMTOKEN | java.lang.String or a custom class
SOAP-ENC:NMTOKENS | java.lang.String or a custom class
SOAP-ENC:Name | java.lang.String
SOAP-ENC:NCName | java.lang.String
SOAP-ENC:ID | java.lang.String
SOAP-ENC:IDREF | java.lang.String
SOAP-ENC:IDREFS | an array or list of java.lang.Strings or a custom class
SOAP-ENC:ENTITY | org.w3c.dom.Entity
SOAP-ENC:ENTITIES | an org.w3c.dom.NodeList containing org.w3c.dom.Entity objects.

These mappings are not written in stone. Some of the XMLish types like SOAP-ENC:ENTITY and SOAP-ENC:IDREFS are particularly uncertain, and may be implemented in different ways in different environments. However, this should give you a pretty good idea of the sorts of mappings that are possible between SOAP types and Java types.

Besides this list of simple types, the http://schemas.xmlsoap.org/soap/encoding/ encoding also defines concepts of structs, references, byte arrays, and arrays.

Structs

A struct is just an element that contains child elements but no mixed content. For example, this is a Quote struct that contains Symbol and Price members:

<Quote xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
  <Symbol>RHAT</Symbol>
  <Price>4.12</Price>
</Quote>

In Java terms, by using the http://schemas.xmlsoap.org/soap/encoding/ encoding style, you’re indicating that you would like this element deserialized into an object of type Quote, which has two properties named Symbol and Price. In other words, the class definition looks something like this:

public class Quote {

  public String getSymbol();
  public double getPrice();

}

You may or may not have such a class in your system. If the SOAP request began its life as a Quote object which was then converted to XML, transmitted across the Internet, and then turned back into a Java object, then perhaps you do have such a class. On the other hand, perhaps the object began its life as a C struct or a C++ object; or perhaps it was never anything except an XML document. In these cases there may not be a convenient Quote class into which you can deserialize this compound object. Another possibility is to decode the name-value pairs into some form of Hashtable or HashMap. The names of the fields would be the keys and the values of the fields would be the values.

What this encoding really tells you is roughly how the author intended this document to be handled. However if you have some other way of making sense of this data, you can use it. You are not limited to any one deserialization form.

References

A reference type uses an href attribute to point to a value stored elsewhere in the SOAP request. This mirrors the structure when two objects both have to contain the same object. For example, consider this trade request:

<Bid xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
  <Symbol>RHAT</Symbol>
  <Price>4.12</Price>
  <Account>777-7777</Account>
</Bid>
<Bid xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
  <Symbol>YHOO</Symbol>
  <Price>4.12</Price>
  <Account>777-7777</Account>
</Bid>

In both cases the account number is the same. Furthermore, it’s not just that the two numbers are equal. They indicate the same object. In Java terms it’s the difference between the equals() method and the == operator. The first tests for equality while the second tests for identity. If the local semantics demand that each Account element be deserialized as an Account object, perhaps with other fields filled in from a database rather than the XML document, then you want some means of saying this document should produce one Account object rather than two. This is done with a reference. Give the first Account element a unique id attribute and use an href attribute in the second element to point to it like this:

<Bid xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
  <Symbol>RHAT</Symbol>
  <Price>4.12</Price>
  <Account id="a1">777-7777</Account>
</Bid>
<Bid xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
  <Symbol>YHOO</Symbol>
  <Price>4.12</Price>
  <Account href="#a1"/>
</Bid>

This document represents two Bid objects. Each has three properties: Symbol, Price, and Account. The Symbols are completely different. The prices are equal but not identical; that is, one can change without changing the other. There are two separate prices here that coincidentally have the same value. The Accounts, however, are identical. There is only one account here, used in two different places.

Arrays

In Java arrays are a funny kind of object, and in SOAP they are too. An array is represented as an element whose type is SOAP-ENC:Array. For example, this is an array of three numbers:

<Bid xsi:type="SOAP-ENC:Array">
   <Price>4.52</Price>
   <Price>0.35</Price>
   <Price>34.68</Price>
 </Bid>

In an array the names of the elements don’t really mean anything. Only the positions matter. If the name of the array doesn’t matter either, you can use a SOAP-ENC:Array element instead. For example, this is an array of three doubles, with no extra information:

<SOAP-ENC:Array>
  <SOAP-ENC:double>4.52</SOAP-ENC:double>
  <SOAP-ENC:double>0.35</SOAP-ENC:double>
  <SOAP-ENC:double>34.68</SOAP-ENC:double>
</SOAP-ENC:Array>

SOAP arrays are not as strongly typed as Java arrays, at least by default. Whereas each array in Java must contain just ints or just strings or just objects, a SOAP array can contain data of varying types. For example, this array contains three items, each with a different name and type:

<Bid xsi:type="SOAP-ENC:Array">
  <Symbol  xsi:type="xsd:token">RHAT</Symbol>
  <Price   xsi:type="xsd:double">4.12</Price>
  <Account xsi:type="xsd:string">777-7777</Account>
</Bid>

Given this possibility, it can be difficult to decode a SOAP array into a Java array. The closest Java equivalent is an Object[] array. However, primitive types like double would need to be replaced by an instance of the matching type wrapper class such as java.lang.Double instead. Another possibility is to use a java.util.Vector or java.util.ArrayList instead of a straight array, though this still doesn’t remove the need for the type wrapper classes.

If you want to restrict the type of array components, you can add a SOAP-ENC:arrayType attribute to the array element. The value of this attribute is the type of the individual component followed by square brackets containing the length of the array. This is more similar to C’s array declaration syntax than Java’s. For example, this array must contain exactly three doubles:

<Bid xsi:type="SOAP-ENC:Array" SOAP-ENC:arrayType="xsd:double[3]">
   <Price>4.52</Price>
   <Price>0.35</Price>
   <Price>34.68</Price>
 </Bid>

Any array component not specifically typed otherwise can be a struct. Furthermore any array component can be another array. However, this does not produce a multidimensional array. Instead, multidimensional arrays are created by stringing together the values from the second row after the values from the first, the values from the third row after the values from the second, and so on. The SOAP-ENC:arrayType attribute indicates how many columns there are. For example, this is a three row by two column array of doubles:

<SOAP-ENC:Array SOAP-ENC:arrayType="xsd:double[3,2]">
   <SOAP-ENC:double>1.1</SOAP-ENC:double>
   <SOAP-ENC:double>1.2</SOAP-ENC:double>
   <SOAP-ENC:double>2.1</SOAP-ENC:double>
   <SOAP-ENC:double>2.2</SOAP-ENC:double>
   <SOAP-ENC:double>3.1</SOAP-ENC:double>
   <SOAP-ENC:double>3.2</SOAP-ENC:double>
 </SOAP-ENC:Array>

Although the XML representation is one-dimensional, the Java interpretation is two-dimensional. When deserialized, this forms the following Java array:

double[][] array = {
  {1.1, 1.2},
  {2.1, 2.2},
  {3.1, 3.2}
}

In the interest of efficiency over potentially slow networks, SOAP allows partially transmitted and sparse arrays. A partially transmitted array (also known as a varying array) does not begin with position 0. Instead it begins at a specified index. For instance, it might have ten components indexed from 3 to 12 inclusive. In SOAP you indicate the position a partially transmitted array begins at with a SOAP-ENC:offset attribute. The value of this attribute is the index of the first element in the array enclosed in square brackets. For example, this array begins at 3:

<SOAP-ENC:Array SOAP-ENC:offset="[3]">
  <SOAP-ENC:string>Component 3</SOAP-ENC:string>
  <SOAP-ENC:string>Component 4</SOAP-ENC:string>
  <SOAP-ENC:string>Component 5</SOAP-ENC:string>
  <SOAP-ENC:string>Component 6</SOAP-ENC:string>
  <SOAP-ENC:string>Component 7</SOAP-ENC:string>
  <SOAP-ENC:string>Component 8</SOAP-ENC:string>
  <SOAP-ENC:string>...</SOAP-ENC:string>
</SOAP-ENC:Array>

Java doesn’t have such arrays, though Pascal and some other languages do. In Java you’d probably deserialize such an array by putting null values or zeroes in the places before the beginning of the array.

A sparse array is one in which a very large percentage of the components are 0 or null. In SOAP a sparse array would only pass the non-zero/non-null components. However, when the array was deserialized, these would be filled in with zeroes or nulls. For a sparse array, the number of elements in the array must be specified by a SOAP-ENC:arrayType attribute. The position of each element that is provided is given by a SOAP-ENC:position attribute. For example, this is a 10-element array that only provides the second, third, and fifth elements:

<SOAP-ENC:Array SOAP-ENC:arrayType="xsd:string[10]">
  <SOAP-ENC:string SOAP-ENC:position="[2]">
    2nd component
  </SOAP-ENC:string>
  <SOAP-ENC:string SOAP-ENC:position="[3]">
    3rd component
  </SOAP-ENC:string>
  <SOAP-ENC:string SOAP-ENC:position="[5]">
    5th component
  </SOAP-ENC:string>
</SOAP-ENC:Array>

The equivalent Java code looks like this:

   String[] array = new String[10];
   array[2] = "\n     2nd component\n    ";
   array[3] = "\n     3rd component\n    ";
   array[5] = "\n     5th component\n    ";

Byte Arrays

A byte array is just a string that somehow encodes binary data. The most common such encoding is Base-64. A schema or an xsi:type attribute is needed to identify the encoding. For example, this is a Base-64 encoded byte array that provides an SHA-1 digital signature for a document. The signature is normally 20 bytes, which becomes 56 characters when translated to Base-64:

<SignatureValue>
AgGOvkMdqdKT7QyMuXPsuomkOqqEhGukKkj4Em7OKKQxYzheuseS8Q==
</SignatureValue>

In Java this would normally be deserialized into a byte array.

SOAP Headers

As well as the body of the request, each SOAP document can contain a header. This is not an HTTP header. Rather it is an additional child of the SOAP-ENV:Envelope element, specifically a SOAP-ENV:Header element. If a SOAP request is an envelope, then the body is the letter inside the envelope, and the header is the writing on the outside of the envelope that tells the Post Office where to deliver it to, where to send it back if they can’t deliver it, and how much you paid to get the letter delivered. In other words, a SOAP header provides meta-information about the request.

The sort of meta-information provided varies from request to request and SOAP application to SOAP application. Some things that can be exchanged in headers include:

  • Protocols the server must understand to process the request.

  • A digital signature for the body of the message

  • A schema for the XML application used in the body

  • Credit card info to pay for the processing

  • A public key to be used to encrypt the response

For instance, Example 2.22 shows a bid document in which the header carries credit card information to pay for the request. In this case, the syntax used for the Payment element is specific to the XML application used in the body and even comes from the same namespace.

Example 2.22. A SOAP Request with a digital signature in the header

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" >
  <SOAP-ENV:Header>
    <Payment xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <Name>Elliotte Harold</Name>
      <Issuer>VISA</Issuer>
      <Number>5125456787651230</Number>
      <Expires>2005-12</Expires>
    </Payment>
  </SOAP-ENV:Header>
  <SOAP-ENV:Body>
    <buy id="buy1"
         xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <symbol>MRBA</symbol>
      <shares>100</shares>
      <account>777-7777</account>
    </buy>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Like the SOAP body, the SOAP header can use any XML application it cares to use to encode the data. It is not limited to a fixed vocabulary. Indeed it can use more than one such vocabulary. The SOAP-ENV:Header element can contain multiple child elements from a hodge-podge of different namespaces. Each one of these elements is called a header entry, and may be treated independently of the other header entries. For example, Example 2.23 adds an additional header containing a digital signature for the request body. The syntax used for the Signature element is defined by XML-Signature Syntax and Processing.

Example 2.23. A SOAP Request with two header entries

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" >
  <SOAP-ENV:Header>
    <Payment xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <Name>Elliotte Harold</Name>
      <Issuer>VISA</Issuer>
      <Number>5125456787651230</Number>
      <Expires>2005-12</Expires>
    </Payment>
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
  <SignedInfo>
    <CanonicalizationMethod
    Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
    <SignatureMethod
      Algorithm="http://www.w3.org/2000/09/xmldsig#dsa-sha1" />
    <Reference URI="file://J/xss4j/requestbody.xml">
    <DigestMethod
      Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
      <DigestValue>3UxhLrdPpK3faRms5FOS6kAoeZI=</DigestValue>
    </Reference>
  </SignedInfo>
  <SignatureValue>
    ZeW/PYGT6A9iOqOrbMmeKOq1aQk+ars/QOC95Bj0xYrNAnLo/WK7+g==
  </SignatureValue>
</Signature>
  </SOAP-ENV:Header>
  <SOAP-ENV:Body>
    <buy id="buy1"
         xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <symbol>MRBA</symbol>
      <shares>100</shares>
      <account>777-7777</account>
    </buy>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The mustUnderstand attribute

An individual SOAP document tends to be tied pretty closely to the service it plans to talk to. You can’t send a request for a stock quote to a server that’s designed to provide basketball scores and expect to get sensible results back. In order that a SOAP request can indicate what is required of a server, it can place a SOAP-ENV:mustUnderstand attribute on each header entry. If this attribute has the value 1, then the service which receives the SOAP request must process the header entry. If it cannot, whether because it does not understand the header entry or for some other reason, then it must fail the request and return a fault. If the SOAP-ENV:mustUnderstand attribute has the value 0, then processing the header is optional. The service should do it if it can, but failing to do so does not automatically lead to a fault. The default is 0.

For example, Example 2.24 is a BUY order that requires the receiver to understand the Payment header. If the server does not recognize that header, it must not attempt to fulfill the order.

Example 2.24. A SOAP Request with a mustUnderstand attribute

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" >
  <SOAP-ENV:Header>
    <Payment xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/"
             SOAP-ENV:mustUnderstand="1">
      <Name>Elliotte Harold</Name>
      <Issuer>VISA</Issuer>
      <Number>5125456787651230</Number>
      <Expires>2005-12</Expires>
    </Payment>
  </SOAP-ENV:Header>
  <SOAP-ENV:Body>
    <buy xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <symbol>MRBA</symbol>
      <shares>100</shares>
      <account>777-7777</account>
    </buy>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The actor attribute

Although this book mostly focuses on SOAP messages that go straight from the sender system to the receiver who will process it, not all systems are this simple. A SOAP message can be forwarded from one SOAP processor to the next until it reaches its ultimate destination. By default, the headers are only read by the last processor. However, you can indicate that a header is intended for a closer processor by using an actor attribute on the header entry element. The value of this attribute is a URI identifying the processor the header entry is intended for.

When a processor receives a SOAP message, it searches the header for header entries addressed to it. It acts on and deletes these header entries. It may also add new header entries intended for processors later in the chain. Then it forwards the message on to the next processor in the chain.

All processors except the last one only act on header entries that are specifically addressed to them. After acting on an entry, the processor deletes it before forwarding the request to the next processor in the chain. Furthermore, the URL http://schemas.xmlsoap.org/soap/actor/next indicates that the header entry should be processed and deleted by the first processor that sees it. The last processor in the chain will process any header entries that are not addressed to any processor in particular as well as any header entries that are addressed specifically to it.

The exact scheme for forwarding SOAP messages from one processor to the next is system dependent. For instance, you might set up a gateway server outside the firewall that verified certain characteristics of a SOAP message before it forwarded it to a processor inside the firewall. Such a gateway would either block or forward each message. A switching processor might inspect the body of the message and forward the request to different SOAP processors depending on what it saw there. Some systems might even use routing included in the messages themselves.

SOAP Limitations

Regrettably, in my opinion, SOAP does not allow developers to take full advantage of XML’s expressiveness and extensibility. First of all, According to the SOAP 1.1 specification, “A SOAP message MUST NOT contain a Document Type Declaration.” This allows non-validating parsers and parsers that cannot resolve external entities to be used to process SOAP messages without concern that they may be misinterpreting them because they don’t apply default namespaces or resolve external entities. However, it also means the document can’t be validated against a DTD.

Also according to the SOAP 1.1 specification, “A SOAP message MUST NOT contain Processing Instructions.” Honestly this makes no sense to me whatsoever. I see little reason for forbidding these. This does mean that all information in a SOAP request must be passed through the defined SOAP structure, but it also means that it’s difficult to include other useful features beyond the SOAP structure. The most obvious is that you can’t easily apply a style sheet to a SOAP document, though that’s not a huge loss since SOAP documents aren’t meant for humans to read in the first place. However, it also means it’s difficult to serve SOAP documents out of the Cocoon application server. There are probably many other environment specific instances where this becomes inconvenient.

Validating SOAP

SOAP is actively hostile to DTDs. The SOAP specification specifically forbids a SOAP request from containing a document type declaration. Thus you really have to use a schema to validate your documents, if you validate them at all.

Unlike XML-RPC, SOAP does have an official schema. In fact it has two, which you can download from the SOAP namespace URLs. The envelope schema at http://schemas.xmlsoap.org/soap/envelope/ describes the SOAP complex types: SOAP-ENV:Envelope, SOAP-ENV:header, SOAP-ENV:Body, etc. The encoding schema at http://schemas.xmlsoap.org/soap/encoding/ defines the SOAP data types shown in Table 2.2: SOAP-ENC:int, SOAP-ENC:NMTOKENS, SOAP-ENC:gYear, SOAP-ENC:Array, etc. These schemas are a little too long to reproduce here, but you can find them in Appendix B.

XML-RPC is a monolithic XML application not designed to be integrated with other XML applications. SOAP, by contrast, is incomplete without some other XML application to form the body of the SOAP request. Thus the SOAP schema cannot be monolithic. It must rely on some other XML application in its own namespace (or perhaps no namespace at all, though this is not recommended) so the SOAP schema cannot on its own validate any SOAP documents. You also need to provide a separate schema for the bodies of your documents, and then you need to merge the two together using xsd:import elements.

For example, Example 2.25 is a master schema for quote request documents such as Example 2.15. This schema declares no elements of its own but does import both SOAP schemas, as well as the schema for getQuote elements seen earlier in Example 2.21. This schema can be used to validate a complete SOAP request that has a getQuote body element. If you wanted to validate the other SOAP documents in this chapter that use other elements in the header and body, you’d just need to write declarations for those elements too. They could be placed in the master schema, trading.xsd, or their own schema documents, whichever seems most convenient.

Example 2.25. A Master Schema for SOAP Trading documents

<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  targetNamespace="http://schemas.xmlsoap.org/soap/envelope/">

  <!-- Standard SOAP schemas -->
  <xsd:include
    schemaLocation="http://schemas.xmlsoap.org/soap/envelope/"
  />
  <xsd:import
    schemaLocation="http://schemas.xmlsoap.org/soap/encoding/"
    namespace="http://schemas.xmlsoap.org/soap/encoding/"
  />

  <!-- Local schema -->
  <xsd:import schemaLocation="trading.xsd"
    namespace="http://namespaces.cafeconleche.org/xmljava/ch2/"
  />

</xsd:schema>

Prev | Up | Next
XML-RPC | Home | Custom Protocols
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 20, 2001


Custom Protocols
Prev Chapter 2. XML Protocols: XML-RPC and SOAP Next

Custom Protocols

XML-RPC is vastly underpowered for many tasks (for instance, any that require strings in a language other than English) whereas SOAP is perhaps overcomplicated for many jobs. There is, however, a convenient middle ground. The really key idea of both SOAP and XML-RPC is that you can pass an XML document over the web using HTTP POST. The exact format of that data doesn’t really matter much as long as both client and server agree on what it is and what it means.

For example, if an online broker such as Ameritrade wanted to provide its clients specialized trading software, it could use any XML format it wanted to send the data back and forth. It might choose to ignore the SOAP envelope completely and just send the getQuote and Buy elements I’ve been placing in the body of the SOAP requests.

There are still a lot of advantages to using a standard format for exchanging data rather than rolling your own every time you need one. Most importantly, a standard format like XML-RPC or SOAP makes it much easier for other developers to integrate their systems with yours. Furthermore, if your system can be easily modelled as a set of procedure calls, then choosing XML-RPC or SOAP allows you to pick from a variety of implementations that take care of a lot of the details for you. Therefore, you shouldn’t go the custom protocol route without some serious thought as to whether it’s truly appropriate. Nonetheless, if a custom XML vocabulary does seem to suit your problem, feel free to define one.

Prev | Up | Next
SOAP | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 22, 2002


Summary
Prev Chapter 2. XML Protocols: XML-RPC and SOAP Next

Summary

XML protocols allow widely distributed systems to communicate with each other over standard HTTP. These protocols have many advantages compared to traditional RPC systems such as CORBA and RMI including simplicity, transparency, and platform independence. Using HTTP allows clients and servers to run on very disparate systems. Using XML allows the data exchanged to achieve any necessary level of complexity. There are very few situations where XML does not provide an obvious means of encoding information. Using schemas even allows XML’s fundamentally text-based format to assign data types to elements so that non-text data such as floating point numbers and dates can be exchanged.

Some XML protocols are based on custom XML applications. Slashdot’s backslash is one example. Others use standard formats like RSS, XML-RPC, and SOAP. RSS is limited to distributing news headlines and summaries. XML-RPC is more general, allowing the transmission of method calls and arguments to remote systems that invoke those methods and return a value. SOAP is the most general of all, allowing the exchange of XML elements of arbitrary complexity which may or may not be treated as method calls and arguments.

Throughout the rest of this book, we’re going to build multiple servers and clients for RSS, XML-RPC, SOAP, and custom systems. We’ll use Java’s networking classes to hide the fact that the XML documents we’re working with are coming from and going to the network. we’re going to focus on the contents of those documents, and how you access and manipulate those contents from inside your own Java programs.

Prev | Up | Next
Custom Protocols | Home | Chapter 3. Writing XML with Java
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 20, 2001


Chapter 3. Writing XML with Java
Prev Next

Chapter 3. Writing XML with Java

Table of Contents

Fibonacci Numbers
Writing XML
Better Coding Practices
Attributes
Producing Valid XML
Namespaces
Output Streams, Writers, and Encodings
A Simple XML-RPC Client
A Simple SOAP Client
Servlets
Summary

No one ever believes me when I tell them how easy it is to develop programs that write XML documents. In fact, writing a program to output an XML document is unbelievably trivial. It’s something an eight-year old typing BASIC in their first class at computer camp can do. In Java, it’s even easier than that due to Java’s strong Unicode support. You don’t need to know any special APIs like DOM or SAX or JDOM. All you have to know is how to System.out.println(). If you want to store your XML document in a file, you can use the FileOutputStream class instead. If you want to serve the document dynamically over a network, it helps to know something about servlets; but in the end it all reduces to writing bytes onto an output stream.

In this chapter, I’m going to develop a program that writes Fibonacci numbers into an XML document. I chose this example because the Fibonacci numbers are a well-known series that’s very easy to generate algorithmically so the examples will be nicely self-contained. However, the principles of XML you learn here will be much more broadly applicable to other, more complex systems. The key idea is that data arrives from some source, is encoded in XML, and is then output. Where the input comes from, whether an algorithm, a file, a network socket, user input, or some other source, really doesn’t concern us here.

I’m going to show you how to use classes you’re already familiar with like OutputStreamWriter, String, and HTTPServlet to generate XML documents. I am going to beat this idea into the ground until you are absolutely convinced that there is nothing special about XML documents that requires any fancy tricks to produce them. Once you’ve finished this chapter you’ll be thoroughly immunized against the snake-oil peddlers who want to sell you multi-hundred thousand dollar software to do what you can do with your existing systems for free.

Fibonacci Numbers

As far as we know, the Fibonacci series was first discovered by Leonardo of Pisa around 1200 C.E. Leonardo was trying to answer the question, “Quot paria coniculorum in uno anno ex uno pario germinatur?”, or, in English, “How many pairs of rabbits are born in one year from one pair?” To solve his problem, Leonardo estimated that rabbits have a one month gestation period, and can first mate at the age of one month, so that each female rabbit has its first litter at two months. He made the simplifying assumption that each litter consisted of exactly one male and one female.

Leonardo begins with one pair of baby rabbits, a male and a female. At the end of the first month, these two have reached puberty and mate. There’s still one pair of rabbits. At the end of the second month, the female gives birth to a new pair of rabbits. There are now two pairs of rabbits, one pair of adults and one pair of babies. The adult pair mates again, so that they will produce one more pair at the end of the third month, at which point there are now three pairs of rabbits. One of these pairs has just been born, but the other two are old enough to mate, which, being rabbits, they do. At the end of the third month, two of the three pairs have babies producing five pairs of rabbits. Meanwhile all rabbits born in previous months mate, so that at the end of the fourth month there will be three more pairs of rabbits. Leonardo realized that the number of pairs at the end of each month was the sum of the number of pairs the preceding month and the number of pairs the month before that. The rabbits don’t simply double in population each month because it takes two months before a rabbit can have its first litter. Nonetheless, the numbers do grow only slightly more slowly than exponentially; and the process continues indefinitely, at least until you run out of rabbit food or the rabbits take over the world, whichever comes first. The number of pairs each month—1, 1, 2, 3, 5, 8, 13, ...— has come to be known as the Fibonacci series after Leonardo’s Latin nickname, Fibonacci (short for filius Bonacci, son of Bonacci).

The rabbits aren’t so important, but the math is. Each integer in the series is formed by the sum of the two previous integers. The first two integers in the series are 1 and 1.[1] The Fibonacci series turns up in some very unexpected places including decimal expansions of π, the golden ratio, the Mandelbrot set, the number of petals on many flowers, the number of spirals on pine cones and pineapples, the number of chambers in a Nautilus shell, and many more.

It’s very easy to calculate the Fibonacci sequence by computer. A simple for loop will do. For example, this code fragment prints the first 40 Fibonacci numbers:

    int low = 1;
    int high = 1;
    for (int i = 1; i < 40; i++) {
      System.out.println(low);
      int temp = high;
      high = high+low;
      low = temp;
    }

However, the Fibonacci numbers do grow very large very quickly, and exceed the bounds of an int shortly before the fiftieth generation. Consequently, it’s better to use the java.math.BigInteger class instead, as Example 3.1 demonstrates:

Example 3.1. A program that calculates the Fibonacci numbers

import java.math.BigInteger;

public class FibonacciNumbers {

  public static void main(String[] args) {

    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;
    for (int i = 1; i <= 10; i++) {
      System.out.println(low);
      BigInteger temp = high;
      high = high.add(low);
      low = temp;
    }

  }

}

When run, this program produces the first ten Fibonacci numbers:

C:\XMLJAVA>java FibonacciNumbers
1
1
2
3
5
8
13
21
34
55

It would be straightforward to read the number of Fibonacci numbers to generate from the command line. However, since user interfaces aren’t the focus of this book, I don’t want to obscure the important parts with too much extraneous fluff.

[1] In some variations, the first two integers are 0 and 1. Aside from the initial zero, this produces the same series.

Prev | Up | Next
Summary | Home | Writing XML
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 21, 2002


Writing XML
Prev Chapter 3. Writing XML with Java Next

Writing XML

Now suppose you don’t just want to dump out a bunch of raw numbers. Instead you want to produce a well-formed XML document such as Example 3.2.

Example 3.2. The first 10 Fibonacci numbers in an XML document

<?xml version="1.0"?>
<Fibonacci_Numbers>
  <fibonacci>1</fibonacci>
  <fibonacci>1</fibonacci>
  <fibonacci>2</fibonacci>
  <fibonacci>3</fibonacci>
  <fibonacci>5</fibonacci>
  <fibonacci>8</fibonacci>
  <fibonacci>13</fibonacci>
  <fibonacci>21</fibonacci>
  <fibonacci>34</fibonacci>
  <fibonacci>55</fibonacci>
</Fibonacci_Numbers>

To produce this, just add string literals for the <fibonacci> and </fibonacci> tags inside the print statements, as well as a few extra print statements to produce the XML declaration and the root element start- and end-tags. XML documents are just text, and you can output them any way you’d output any other text document. Example 3.3 demonstrates.

Example 3.3. A program that outputs the Fibonacci numbers as an XML document

import java.math.BigInteger;

public class FibonacciXML {

  public static void main(String[] args) {

      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      System.out.println("<?xml version=\"1.0\"?>");
      System.out.println("<Fibonacci_Numbers>");
      for (int i = 0; i < 10; i++) {
        System.out.print("  <fibonacci>");
        System.out.print(low);
        System.out.println("</fibonacci>");
        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }
      System.out.println("</Fibonacci_Numbers>");

  }

}

Better Coding Practices

Although I’m going to to keep the examples as simple and small as I possibly can in this book, and save my lines of code for the points that are relevant to XML, you shouldn’t use that as an excuse to forget all the good programming practices you’ve learned in the past, even if I don’t use them here. For instance, keeping too much data in string literals throughout the code makes a program very hard to localize and maintain. You may wish to move element names and other constants into final static fields as shown in Example 3.4. You might even want to read the values of these variables from a resource bundle or system properties so they can be edited independently of the source code. I’m not going to do this here, because for purposes of education it’s more important to have short, self-contained examples that make the key points easy to grasp. However, in larger, more complex programs these techniques are essential.

Example 3.4. Using named constants for element names

import java.math.BigInteger;

public class FibonacciConstants {

  public final static String rootElementName
   = "Fibonacci_Numbers";
  public final static String fibonacciElementName = "fibonacci";
  public final static String xmlDeclaration
   = "<?xml version=\"1.0\"?>";

  public static void main(String[] args) {

      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      System.out.println(xmlDeclaration);
      System.out.println("<" + rootElementName + ">");
      for (int i = 0; i < 10; i++) {
        System.out.print("  <" + fibonacciElementName +">");
        System.out.print(low);
        System.out.println("</" + fibonacciElementName +">");
        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }
      System.out.println("</" + rootElementName + ">");

  }

}

Attributes

Now let’s suppose you want to add some attributes to the elements. For instance, you might want to give each fibonacci element an index attribute specifying which Fibonacci number it is (the first, the second, the third, and so forth).

To do this, just add the extra strings to the source code, and calculate the index value from the loop index. Example 3.5 demonstrates.

Example 3.5. A Java program that writes an XML document that uses attributes

import java.math.BigInteger;

public class FibonacciAttributes {

  public static void main(String[] args) {

      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      System.out.println("<?xml version=\"1.0\"?>");
      System.out.println("<Fibonacci_Numbers>");
      for (int i = 1; i <= 10; i++) {
        System.out.print("  <fibonacci index=\"" + i + "\">");
        System.out.print(low);
        System.out.println("</fibonacci>");
        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }
      System.out.println("</Fibonacci_Numbers>");

  }

}

The only even remotely difficult part of this is realizing you have to escape the double quotes used to delimit the attribute values as \" in the Java source code. An alternative is to use single quotes to delimit the attribute values. These don’t need to be escaped inside Java string literals. For example,

        System.out.print("  <fibonacci index='" + i + "'>");

Here’s the output from Example 3.5:

<?xml version="1.0"?>
<Fibonacci_Numbers>
  <fibonacci index="1">1</fibonacci>
  <fibonacci index="2">1</fibonacci>
  <fibonacci index="3">2</fibonacci>
  <fibonacci index="4">3</fibonacci>
  <fibonacci index="5">5</fibonacci>
  <fibonacci index="6">8</fibonacci>
  <fibonacci index="7">13</fibonacci>
  <fibonacci index="8">21</fibonacci>
  <fibonacci index="9">34</fibonacci>
  <fibonacci index="10">55</fibonacci>
</Fibonacci_Numbers>

Producing Valid XML

So far the Java programs have produced XML documents that were well-formed but not valid. Making them valid is not difficult. Simply print a a document type declaration that either includes an internal DTD subset or points to an appropriate external DTD subset. Example 3.6 takes the internal subset approach:

Example 3.6. A Java program that generates a valid document

import java.math.BigInteger;

public class ValidFibonacci {

  public static void main(String[] args) {

      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      System.out.println("<?xml version=\"1.0\"?>");
      System.out.println("<!DOCTYPE Fibonacci_Numbers [");
      System.out.println(
       "  <!ELEMENT Fibonacci_Numbers (fibonacci)*>");
      System.out.println("  <!ELEMENT fibonacci (#PCDATA)>");
      System.out.println(
       "  <!ATTLIST fibonacci index NMTOKEN #REQUIRED>");
      System.out.println("]>");
      System.out.println("<Fibonacci_Numbers>");
      for (int i = 0; i < 10; i++) {
        System.out.print("  <fibonacci index=\"" + i + "\">");
        System.out.print(low);
        System.out.println("</fibonacci>");
        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }
      System.out.println("</Fibonacci_Numbers>");

  }

}

Here’s the output from this program, including the document type declaration:

<?xml version="1.0"?>
<!DOCTYPE Fibonacci_Numbers [
  <!ELEMENT Fibonacci_Numbers (fibonacci)*>
  <!ELEMENT fibonacci (#PCDATA)>
  <!ATTLIST fibonacci index NMTOKEN #REQUIRED>
]>
<Fibonacci_Numbers>
  <fibonacci index="0">1</fibonacci>
  <fibonacci index="1">1</fibonacci>
  <fibonacci index="2">2</fibonacci>
  <fibonacci index="3">3</fibonacci>
  <fibonacci index="4">5</fibonacci>
  <fibonacci index="5">8</fibonacci>
  <fibonacci index="6">13</fibonacci>
  <fibonacci index="7">21</fibonacci>
  <fibonacci index="8">34</fibonacci>
  <fibonacci index="9">55</fibonacci>
</Fibonacci_Numbers>

Attaching a schema is no harder. Just place the necessary xmlns:xsi and xsi:noNamespaceSchemaLocation attributes on the root element.

Namespaces

Suppose instead of using a custom Fibonacci number vocabulary, you want to use the standard MathML vocabulary as shown in Example 3.7. The element names all have prefixes and an xmlns:mathml attribute on the root element binds the mathml prefix to the http://www.w3.org/1998/Math/MathML namespace URI. Each Fibonacci number is included in a mathml:mrow divided into a mathml:mi element, a mathml:mo element, and a mathml:mn element.

Example 3.7. A MathML document containing Fibonacci numbers

<?xml version="1.0"?>
<mathml:math xmlns:mathml="http://www.w3.org/1998/Math/MathML">
  <mathml:mrow>
    <mathml:mi>f(1)</mathml:mi>
    <mathml:mo>=</mathml:mo>
    <mathml:mn>1</mathml:mn>
  </mathml:mrow>
  <mathml:mrow>
    <mathml:mi>f(2)</mathml:mi>
    <mathml:mo>=</mathml:mo>
    <mathml:mn>1</mathml:mn>
  </mathml:mrow>
  <mathml:mrow>
    <mathml:mi>f(3)</mathml:mi>
    <mathml:mo>=</mathml:mo>
    <mathml:mn>2</mathml:mn>
  </mathml:mrow>
  <mathml:mrow>
    <mathml:mi>f(4)</mathml:mi>
    <mathml:mo>=</mathml:mo>
    <mathml:mn>3</mathml:mn>
  </mathml:mrow>
</mathml:math>

The markup is somewhat more complex, but the Java code is not significantly more so as Example 3.8 demonstrates.

Example 3.8. A Java program that generates a MathML document

import java.math.BigInteger;

public class MathMLFibonacci {

  public static void main(String[] args) {

      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      System.out.println("<?xml version=\"1.0\"?>");
      System.out.println(
        "<mathml:math "
        + "xmlns:mathml=\"http://www.w3.org/1998/Math/MathML\">"
      );
      for (int i = 1; i <= 10; i++) {
        System.out.println("  <mathml:mrow>");
        System.out.println("    <mathml:mi>f(" + i
         + ")</mathml:mi>");
        System.out.println("    <mathml:mo>=</mathml:mo>");
        System.out.println("    <mathml:mn>" + low
         + "</mathml:mn>");
        System.out.println("  </mathml:mrow>");
        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }
      System.out.println("</mathml:math>");

  }

}

I could continue, showing you how to add comments, processing instructions, CDATA sections and other features of XML; but I think by now you’re getting the idea. XML documents are just text. They can be represented inside Java programs as various combinations of String literals and String variables. Other data types like int and BigInteger can be converted to their string representations either implicitly by concatenating them with strings or explicitly by invoking methods such as toString() and String.valueOf(). The fact that XML documents are text makes it very easy to organize XML output this way.

When you use this technique, you are responsible for following all the well-formedness, namespace well-formedness, and validity rules of XML. Nothing prevents you from producing incorrect XML. In fact, your first few efforts are likely to be malformed. You’ll want to run your output through a tool such as Xerces’s sax.Counter to check it for well-formedness and perhaps validity. Making sure the output is correct is simply one part of testing and debugging your code. The same would still be true if you were outputting a non-XML format. In fact, if anything, XML makes these tests easier because it’s straightforward to write declarative schemas that express exactly what is and is not legal and to compare your output against these schemas.

Eventually we’ll take up some alternatives to the direct string approach such as DOM and JDOM that do allow you to automatically maintain well-formedness and sometimes even validity. However, for many simple cases, these are vast overkill. It can be much simpler to just write a few strings onto an output stream. In the next section, we’ll look at how you can change the characteristics of the output stream you’re writing on.

Prev | Up | Next
Chapter 3. Writing XML with Java | Home | Output Streams, Writers, and Encodings
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified August 10, 2001


Output Streams, Writers, and Encodings
Prev Chapter 3. Writing XML with Java Next

Output Streams, Writers, and Encodings

Most of the time you don’t want to dump an XML document to System.out. Instead you want to write it in a file or onto a network socket. You might even store it in a string and pass it to some other process. What connects all these possible targets in Java is the java.io.OutputStream class. Files, sockets, and even strings can all be treated as just another kind of stream.

XML documents are text. The text is made up of Unicode characters. When those Unicode characters are actually written onto a stream, you need to pick a character encoding that specifies how each character is converted into bytes. This encoding can be one of the Unicode encodings such as UTF-8 or UTF-16 or it can be a local code page such as ISO-8859-1 or MacRoman. Characters that don’t exist in the local code page can be escaped using numeric character references. The encoding declaration will be set to indicate the character set in use. The normal way Java converts characters into bytes in a specific encoding is by chaining an OutputStreamWriter to an OutputStream. As chars and strings are written onto the OutputStreamWriter, they are converted to bytes in the specified encoding which are then written onto the underlying OutputStream.

Let’s suppose you want to dump the Fibonacci numbers into a file called fibonacci.xml in the current working directory. First you would open a FileOutputStream to that file, like this:

OutputStream fout = new FileOutputStream("fibonacci.xml");

If performance is at all a concern, you would immediately chain this FileOutputStream to a BufferedOutputStream like this:

OutputStream bout = new BufferedOutputStream(fout);

Then you would chain the BufferedOutputStream to an OutputStreamWriter. You’d pass the Java name of the encoding you want as the second argument to the OutputStreamWriter() constructor. For example, this line chooses the ISO-8859-1, Latin-1 encoding, though it uses Java’s name for this encoding, “8859_1”:

OutputStreamWriter out = new OutputStreamWriter(bout, "8859_1");

Finally, you’d write the output onto that OutputStreamWriter, making sure to include the right encoding declaration using the XML name for the encoding. Example 3.9 demonstrates.

Example 3.9. A Java program that writes an XML file

import java.math.BigInteger;
import java.io.*;

public class FibonacciFile {

  public static void main(String[] args) {

      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      try {
        OutputStream fout= new FileOutputStream("fibonacci.xml");
        OutputStream bout= new BufferedOutputStream(fout);
        OutputStreamWriter out
         = new OutputStreamWriter(bout, "8859_1");

        out.write("<?xml version=\"1.0\" ");
        out.write("encoding=\"ISO-8859-1\"?>\r\n");
        out.write("<Fibonacci_Numbers>\r\n");
        for (int i = 1; i <= 10; i++) {
          out.write("  <fibonacci index=\"" + i + "\">");
          out.write(low.toString());
          out.write("</fibonacci>\r\n");
          BigInteger temp = high;
          high = high.add(low);
          low = temp;
        }
        out.write("</Fibonacci_Numbers>\r\n");

        out.flush();  // Don't forget to flush!
        out.close();
      }
      catch (UnsupportedEncodingException e) {
        System.out.println(
         "This VM does not support the Latin-1 character set."
        );
      }
      catch (IOException e) {
        System.out.println(e.getMessage());
      }

  }

}

One change from the System.out version is that the line breaks have to be encoded explicitly. They’re not really necessary for this XML document, but the examples are prettier if the XML isn’t just one long line of text. I recommend using a carriage-return/linefeed pair (\r\n) as your line break. This is the native format for DOS and Windows, and most Unix and Macintosh text editors can handle it. More importantly it is the standard line ending for network protocols such as HTTP and SMTP. XML parsers do normalize all line breaks to a single linefeed on input, so the proper choice of line break for an XML document is not nearly as fraught as for some other types of files. Nonetheless, picking carriage-return/linefeed does help when processing or transmitting XML documents with non-XML-aware tools.

Although most XML parsers written in Java support exactly those encodings that are available in Java, they don’t use the same names. Java tends to use underscores where XML uses hyphens in encoding names, or to eliminate the hyphens completely. The reason is that earlier Java virtual machines used reflection to locate the classes that convert between different encodings, so its encoding names had to be legal Java class names. Table 3.1 lists the Java and XML equivalents of the standard character sets and encodings. Later versions of Java often allow multiple names for the same encoding, especially in Java 1.4. Here I’ve picked the names that are supported across the broadest range of virtual machines.

Table 3.1. Standard Character Sets and Encodings

XML Name | Java Name | First supported in Java | Scripts and Languages
ISO-8859-1 | 8859_1 | 1.1 | Latin-1: ASCII plus the accented characters needed for most Western European languages including Albanian, Basque, Breton, Catalan, Cornish, Danish, Dutch, English, Estonian, Faroese, Finnish, French, Frisian, Galician, German, Greenlandic, Icelandic, Irish, Italian, Latin, Luxemburgish, Norwegian, Portuguese, Rhaeto-Romanic, Scottish Gaelic, Sorbian, Spanish, and Swedish as well as many non-European languages written in the Latin alphabet such as Swahili and Malaysian
ISO-8859-2 | 8859_2 | 1.1 | Latin-2: ASCII plus the accented characters needed for most Central European languages including Albanian, Croatian, Czech, Finnish, German, Hungarian, Latin, Polish, Romanian, Slovak, Slovenian, and Sorbian
ISO-8859-3 | 8859_3 | 1.1 | Latin-3: ASCII plus the accented characters needed for most Southern European languages including English, Esperanto, Finnish, French, German, Italian, Latin, Maltese, Portuguese, and Turkish
ISO-8859-4 | 8859_4 | 1.1 | Latin-4: ASCII plus the accented characters needed for most Northern European languages including Danish, English, Estonian, Finnish, German, Greenlandic, Latin, Latvian, Lithuanian, Norwegian, S�mi, Slovenian, and Swedish
ISO-8859-5 | 8859_5 | 1.1 | ASCII plus Cyrillic
ISO-8859-6 | 8859_6 | 1.1 | ASCII plus Arabic
ISO-8859-7 | 8859_7 | 1.1 | ASCII plus Greek
ISO-8859-8 | 8859_8 | 1.1 | ASCII plus Hebrew
ISO-8859-9 | 8859_9 | 1.1 | Latin-5: same as Latin-1 except the Turkish letters Ğ, ğ, İ, ı, Ş, and ş take the place of the Icelandic letters þ, Þ, ý, Ý, Ð, and ð
ISO-8859-13 | ISO8859_13 | 1.3 | Latin-7: ASCII plus the accented characters needed for most Baltic languages including Latvian, Lithuanian, Estonian, and Finnish, as well as English, Danish, Swedish, German, Slovenian, and Norwegian.
ISO-8859-15 | ISO8859_15_FDIS | 1.2 | Latin-9: same as Latin-1 but with the Euro sign € instead of the international currency symbol ¤. It also replaces the infrequently used symbol characters ¦, ¨, ´, ¸, ¼, ½, and ¾ with the infrequently used French and Finnish letters Š, š, Ž, ž, Œ, œ, and Ÿ.
UTF-8 | UTF8 | 1.1 | The default encoding of XML documents; each Unicode character is represented in between 1 and 4 bytes.
UTF-16 | UnicodeBig or UnicodeLittle | 1.2 | An encoding of Unicode in which characters in the Basic Multilingual Plane are encoded in two bytes, and all other characters are encoded as two two-byte surrogates
ISO-10646-UCS-2 | N/A | N/A | A straightforward encoding in which each Unicode character is represented as a two-byte integer; cannot represent characters outside the Basic Multilingual Plane
ISO-10646-UCS-4 | N/A | N/A | A straightforward encoding in which each Unicode character is represented as a four-byte integer
ISO-2022-JP | JIS | 1.1 | Japanese
Shift_JIS | SJIS | 1.1 | Japanese
EUC-JP | EUCJIS | 1.1 | Japanese
US-ASCII | ASCII | 1.2 | English
GBK | GBK | 1.1 | Simplified Chinese
Big5 | Big5 | 1.1 | Traditional Chinese
ISO-2022-CN | ISO2022CN | 1.1 | Traditional Chinese
ISO-2022-KR | ISO2022KR | 1.1 | Korean

I have deliberately omitted XML legal encodings that are not yet supported by Java such as ISO-8859-10, ISO-8859-11, ISO-8859-14, and ISO-8859-16. It’s not hard to add them in Java 1.4; but since they’re not available by default, you’re better off picking UTF-8 or one of the other encodings of Unicode.

The exact list of encodings Java supports varies from virtual machine to virtual machine and version to version. Java 1.4 is a major leap forward in support for many character sets, as well as for different aliases for character set names. However, since the standard Unicode and ISO encodings let you handle most environments today, there’s little reason to use other encodings in XML documents.

Some parsers, including Xerces-J, have an option to recognize the Java names, as well as all the other encodings Java supports. This can be useful when you’re reading XML documents sent to you by other people and systems. However, you should not generate documents that use these encodings. The standard encodings in Table 3.1 should be sufficient for any document you need to create, and are a lot more cross-platform compatible than platform-specific code pages such as Cp1252 and MacRoman. In a few cases you might prefer to use standard non-Unicode, non-ISO national character sets such as KS C 5601 for Korean or KOI8-R for Cyrillic. These are OK too, but still a little less well recognized around the world than the standard encodings shown in Table 3.1. The general principle is to be conservative in what you generate, but liberal in what you accept. Try to stick to the most standard encodings when writing documents, but accept any encoding you recognize when reading documents.

Other character sets you should not use in XML but that are available in Java include UTF-16BE and UTF16-LE. These are big-endian and little-endian encodings of Unicode without an explicit byte order mark. XML documents in UTF-16 must have an explicit byte order mark. It may not be omitted. UTF-8 documents may have a byte order mark, but in general should not for maximum compatibility with other software.

Note

Output streams, output stream writers, files, Unicode, character sets and character encodings, and many other aspects of input and output in Java are covered in much more detail in my book [Java I/O, O’Reilly & Associates, 1999, ISBN 1-56592-485-1]

Prev | Up | Next
Writing XML | Home | A Simple XML-RPC Client
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 24, 2002


A Simple XML-RPC Client
Prev Chapter 3. Writing XML with Java Next

A Simple XML-RPC Client

An XML-RPC request is just an XML document sent over a network socket. It’s easy to use the techniques of this chapter to write a very simple XML-RPC client for a particular service. You just ask the user for the data to send to the server, wrap it up in some XML, and write it onto a URL object pointing at the server. The response will come back in XML as well. Since we haven’t yet learned how to read XML documents, I’ll just dump the response as text to System.out. Later we’ll pay more attention to the response, and provide a nicer user interface.

The specific XML-RPC service we’re going to talk to is a Fibonacci generator I’ll develop in later chapters. The request passes an int to the server. The server responds with the value of that Fibonacci number. For example, this request document asks for the value of the 23rd Fibonacci number:

<?xml version="1.0"?>
<methodCall>
  <methodName>calculateFibonacci</methodName>
  <params>
    <param>
      <value><int>23</int></value>
    </param>
  </params>
</methodCall>

And here’s the response:

<?xml version="1.0"?>
<methodResponse>
  <params>
    <param>
      <value><double>28657</double></value>
    </param>
  </params>
</methodResponse>

The response is passed back as a double because XML-RPC ints are limited to a maximum size of 2,147,483,647, and as previously noted Fibonacci numbers very quickly exceed that. XML-RPC does not define a maximum size for a double. Some implementations may have maximum sizes, nonetheless. If this is a concern, you can send the response back as a string instead.

The client needs to read an integer input by the user, wrap that integer in the XML-RPC envelope, then send that document to the server using HTTP POST. In Java the simplest way to post a document is through the java.net.HttpURLConnection class. You get an instance of this class by calling a URL object’s openConnection() method, and then casting the resulting URLConnection object to HttpURLConnection. Once you have an HttpURLConnection, you set its protected doOutput field to true using the setDoOutput() method and its request method to POST using the setRequestMethod() method. Then you grab hold of an output stream using getOutputStream() and proceed to write your request body on that. Example 3.10 does all of this.

Example 3.10. Connecting an XML-RPC server with URLConnection

import java.net.*;
import java.io.*;

public class FibonacciXMLRPCClient {

  private static String defaultServer
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println(
       "Usage: java FibonacciXMLRPCClient index serverURL");
      return;
    }

    String index = args[0];

    String server;
    if (args.length <= 1) server = defaultServer;
    else server = args[1];

    try {
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();
      OutputStreamWriter wout = new OutputStreamWriter(out, "UTF-8");

      wout.write("<?xml version=\"1.0\"?>\r\n");
      wout.write("<methodCall>\r\n");
      wout.write(
       "  <methodName>calculateFibonacci</methodName>\r\n");
      wout.write("  <params>\r\n");
      wout.write("    <param>\r\n");
      wout.write("      <value><int>");
      wout.write(index);
      wout.write("</int></value>\r\n");
      wout.write("    </param>\r\n");
      wout.write("  </params>\r\n");
      wout.write("</methodCall>\r\n");

      wout.flush();
      out.close();

      InputStream in = connection.getInputStream();
      int c;
      while ((c = in.read()) != -1) System.out.write(c);
      System.out.println();
      in.close();
      out.close();
      connection.disconnect();
    }
    catch (IOException e) {
      System.err.println(e);
      e.printStackTrace();
    }

  }  // end main

}

You’ll see the servlet that services this client in Chapter 10. In the meantime, if you’d like to test this program you can point it at http://www.elharo.com/fibonacci/XML-RPC where I have the servlet running. Here’s some sample output from that site:

C:\XMLJAVA>java FibonacciXMLRPCClient 10
<?xml version="1.0"?>
<methodResponse>
  <params>
    <param>
      <value><double>55</double></value>
    </param>
  </params>
</methodResponse>
Prev | Up | Next
Output Streams, Writers, and Encodings | Home | A Simple SOAP Client
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 24, 2002


A Simple SOAP Client
Prev Chapter 3. Writing XML with Java Next

A Simple SOAP Client

Talking to a SOAP server that provides Fibonacci numbers is not significantly harder than talking to an XML-RPC server. You just have to adjust the syntax of your request to use SOAP instead of XML-RPC. Again, we’ll talk to a Fibonacci generator I’ll develop in later chapters, but that can be previewed at http://www.elharo.com/fibonacci/SOAP. The body of each request document contains a calculateFibonacci element in the http://namespaces.cafeconleche.org/xmljava/ch3/ namespace. This element contains a single positive integer:

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >
  <SOAP-ENV:Body>
    <calculateFibonacci
      xmlns="http://namespaces.cafeconleche.org/xmljava/ch3/"
      type="xsi:positiveInteger">10</calculateFibonacci>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The server responds with a list of Fibonacci numbers using the vocabulary designed earlier in this chapter, enclosed in the usual SOAP response envelope. For example, here’s the response to a request for the first ten Fibonacci numbers:

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" />
  <SOAP-ENV:Body>
    <Fibonacci_Numbers
      xmlns="http://namespaces.cafeconleche.org/xmljava/ch3/">
      <fibonacci index="1">1</fibonacci>
      <fibonacci index="2">1</fibonacci>
      <fibonacci index="3">2</fibonacci>
      <fibonacci index="4">3</fibonacci>
      <fibonacci index="5">5</fibonacci>
      <fibonacci index="6">8</fibonacci>
      <fibonacci index="7">13</fibonacci>
      <fibonacci index="8">21</fibonacci>
      <fibonacci index="9">34</fibonacci>
      <fibonacci index="10">55</fibonacci>
    </Fibonacci_Numbers>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Conceptually, the SOAP client is almost identical to Example 3.10. It reads one or more integers from the command lines, opens a URLConnection to the server, and POSTs each integer to the server wrapped inside a bunch of XML markup. It then receives a batch of XML markup in response which it prints on System.out. The markup used has changed. However, all the markup is just strings. One string is more or less the same as the next as far as Java is concerned. Example 3.11 shows the completed code.

Example 3.11. Connecting to a SOAP server with URLConnection

import java.net.*;
import java.io.*;

public class FibonacciSOAPClient {

  public final static String DEFAULT_SERVER
   = "http://www.elharo.com/fibonacci/SOAP";
  public final static String SOAP_ACTION
   = "http://www.example.com/fibonacci";

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println(
       "Usage: java FibonacciSOAPClient index URL");
      return;
    }
    String input = args[0];
    String server = DEFAULT_SERVER;
    if (args.length >= 2) server = args[1];

    try {
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;

      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      connection.setRequestProperty("SOAPAction", SOAP_ACTION);

      OutputStream out = connection.getOutputStream();
      Writer wout = new OutputStreamWriter(out);

      wout.write("<?xml version='1.0'?>\r\n");
      wout.write("<SOAP-ENV:Envelope ");
      wout.write("xmlns:SOAP-ENV=");
      wout.write(
        "'http://schemas.xmlsoap.org/soap/envelope/' "
      );
      wout.write("xmlns:xsi=");
      wout.write(
        "'http://www.w3.org/2001/XMLSchema-instance'>\r\n");
      wout.write("  <SOAP-ENV:Body>\r\n");
      wout.write("    <calculateFibonacci ");
      wout.write(
    "xmlns='http://namespaces.cafeconleche.org/xmljava/ch3/'\r\n"
      );
      wout.write("    type='xsi:positiveInteger'>" + input
       + "</calculateFibonacci>\r\n");
      wout.write("  </SOAP-ENV:Body>\r\n");
      wout.write("</SOAP-ENV:Envelope>\r\n");

      wout.flush();
      wout.close();

      InputStream in = connection.getInputStream();
      int c;
      while ((c = in.read()) != -1) System.out.write(c);
      in.close();

    }
    catch (IOException e) {
      System.err.println(e);
    }

  } // end main

} // end FibonacciSOAPClient

The one new piece here is that SOAP 1.1 requires the client to set a SOAPAction field in the HTTP header. This is done with URLConnection’s setRequestProperty() method. Some servers may dispatch the request based on this field without actually parsing the body of the request. This server doesn't pay any attention to the SOAPAction field, however, so I just set it to a reaosnable default value. SOAP 1.2 will make SOAPAction optional.

Prev | Up | Next
A Simple XML-RPC Client | Home | Servlets
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 24, 2002


Servlets
Prev Chapter 3. Writing XML with Java Next

Servlets

On the server side, we’re not ready to accept an XML-RPC or SOAP request yet because that would require parsing an input XML document. I’ll take that up beginning in a couple of chapters. However, we are completely ready to respond to a CGI request with an XML document. Since this is a book about Java, I’ll use servlets to do this. The user will connect to a known URL. The query string component will include a field asking for a certain number of Fibonacci numbers. The servlet will calculate that number and return it as an XML document with an attached stylesheet.

The query string will include a single generations field whose value is the index of the last Fibonacci number you want. For example, if the servlet is installed at http://www.elharo.com/fibonacci, then you’d ask for 42 Fibonacci numbers by loading the URL http://www.elharo.com/fibonacci?generations=42.

The servlet engine handles the HTTP header and all other interactions with the client. If you need to change any of the details, you use the HttpServletRequest and HttpServletResponse objects passed as arguments to the doGet() method (for HTTP GET) or doPost() (for HTTP POST). This example behaves the same for both POST and GET, so the doPost() method will just call the doGet() method.

For input, it’s easy to read the query string field from a servlet using the getParameter() and getParameters() methods. These handle all necessary decoding of x-www-form-urlencoded values. If there’s more than one field in the query string with the specified name, then getParameter() returns the value of the first one. getParameters() returns an array of all the field values of fields with a certain name. In this case, I’ll be satisfied with just the first field named generations so I’ll choose getParameter(). Because the getParameter() method effectively hides whether the query string was included in the URL sent by the GET method or in the body sent by the POST method, adding support for POST to this servlet is very easy; simply implement the doPost() method and pass its argument to the doGet() method.

For output, we just need to set the content type to text/xml and write the response on the PrintWriter returned by the HttpServletResponse’s getWriter() method. By default, this PrintWriter uses the ISO-8859-1 encoding. However, if you want to use another encoding such as UTF-8, you must adjust the MIME type to reflect that. For instance, to send data in UTF-16, you set the MIME type to text/xml; charset=UTF-16. Example 3.12 shows the completed servlet.

Example 3.12. A servlet that generates XML

import javax.servlet.*;
import javax.servlet.http.*;
import java.io.*;
import java.math.BigInteger;

public class FibonacciServlet extends HttpServlet {

  public void doGet(
   HttpServletRequest request, HttpServletResponse response)
   throws ServletException, IOException {

    //read the query string
    int numberOfGenerations = 10;
    String generations = request.getParameter("generations");
    try {
      numberOfGenerations = Integer.parseInt(generations);
    }
    catch (Exception e) { // NumberFormat or NullPointerException
      // use default value of 10
    }

    response.setContentType("text/xml; charset=UTF-8");
    PrintWriter out = response.getWriter();

    out.println("<?xml version=\"1.0\"?>");
    out.print  ("<?xml-stylesheet ");
    out.println(
     "type='text/css' href='/xml/styles/fibonacci.css'?>");
    out.println("<Fibonacci_Numbers>");

    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;
    for (int i = 1; i <= numberOfGenerations; i++) {
      out.print("  <fibonacci index=\"" + i + "\">");
      out.print(low);
      out.println("</fibonacci>");

      BigInteger temp = high;
      high = high.add(low);
      low = temp;
    }
    out.println("</Fibonacci_Numbers>");

    out.close();

  }

  public void doPost(
   HttpServletRequest request, HttpServletResponse response)
   throws ServletException, IOException {

    doGet(request, response);

  }

}

Once you’ve compiled this servlet, you’ll need to install it on your server. Details vary a little from servlet engine to servlet engine. I use the Apache Jakarta Project’s Tomcat, which is the official reference implementation of the Java Servlet API. If you’re using something different, please consult your local documentation.

To install the servlet into Tomcat, first upload the FibonacciServlet.class file into your WEB-INF/classes directory. This makes it accessible at a URL based on the class name such as http://www.cafeconleche.org:8080/servlet/FibonacciServlet. Here http://www.cafeconleche.org:8080 is the URL for my servlet context. (Yours will of course be something different.) servlet is the part of the URL that tells the servlet engine it should expect to load a servlet by its class name, and FibonacciServlet is the name of the servlet class.

If you’d like the servlet to be accessible by a more convenient name, edit the web.xml file in the servlet context directory to map the name you want to the name of the class. For example, the following lines map the FibonacciServlet class to the URL http://www.cafeconleche.org:8080/chapter3:

 <servlet>
  <servlet-name>fibonacci</servlet-name>
  <servlet-class>FibonacciServlet</servlet-class>
 </servlet>

 <servlet-mapping>
   <servlet-name>fibonacci</servlet-name>
   <url-pattern>chapter3</url-pattern>
 </servlet-mapping>

In Tomcat 3.2 and earlier, you must shutdown and restart the servlet engine before Tomcat will notice that the web.xml file has changed and use the new mapping.

Note

For more details about servlets, web.xml, Tomcat, and related topics, you might wish to consult the reference I used when writing this chapter, [Inside Servlets, 2nd edition, by Dustin R. Calloway, (Addison Wesley, 2001) ISBN 0-201-70906-6]

Prev | Up | Next
A Simple SOAP Client | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 21, 2002


Summary
Prev Chapter 3. Writing XML with Java Next

Summary

XML documents are text. This means you can create an XML document using any technique you would normally use to create text files including but not limited to output streams, writers, toString() methods, char arrays, or anything else that produces text. When you do this, you’re responsible for maintaining well-formedness and validity. However, doing this is not hard, certainly no harder than debugging any other part of a program and often considerably easier.

Since XML documents are text, you should use a Writer rather than an OutputStream to generate an XML document. However, since the Writer class doesn’t have any standard method of determining what encoding it’s using, you should normally chain your Writer to an OutputStream you control which you immediately chain to an OutputStreamWriter rather than accepting a Writer of unknown provenance from somewhere else. This is the only way to make sure that the encoding declaration matches what you’re actually outputting.

On the client side, the java.net.URLConnection class is needed to talk to XML-RPC and SOAP servers because it allows you to use the HTTP POST method. The request can be sent just by writing the XML document representing the request onto the URLConnection’s output stream, and then flushing. The response returns as an XML document also. Future chapters will address the parsing of such response documents.

On the server side, Java servlets serve XML just as easily as they serve HTML, You simply have to remember to set the MIME media type of the response to text/xml and include an encoding in the MIME type if you want anything other than Latin-1. You write the XML document onto the servlet’s OutputStream just as you’d write an HTML document. There’s really nothing special about servlets in this respect, other than that this is a book about Java. Similar techniques work for CGIs written in C, AppleScript, or Perl or for other server side programming environments such as ASP and PHP.

Prev | Up | Next
Servlets | Home | Chapter 4. Converting Flat Files to XML
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 24, 2002


Chapter 4. Converting Flat Files to XML
Prev Next

Chapter 4. Converting Flat Files to XML

Table of Contents

The Budget
The Model
Input
Determining the Output Format
Validation
Attributes
Building Hierarchical Structures from Flat Data
Alternatives to Java
Imposing Hierarchy with XSLT
The XML Query Language
Relational Databases
Summary

Relatively little of the world’s data is currently stored in XML. Much of it is stored in flat files as tab-delimited text, comma separated values, or some similar format. More is locked up in databases of one kind or another, whether relational, hierarchical, or object-based. Even more is hidden inside unstructured documents including Microsoft Word files, HTML documents, and plain text. XML tools are not suitable for working with any of this.

There are no magic bullets that will convert all your data to semantically tagged XML. There are a few specialized programs that convert certain formats such as Word documents to particular XML applications such as XHTML. However, the output from even the best of these tools often needs to be cleaned up by hand. How much clean-up work you need to do generally depends on how structured the data format is to start with and how clean the data is. It’s relatively easy to encode a relational table from a DB2 database as XML because it already has a lot of structure and a mandatory schema. It’s a lot harder in practice to convert tab-delimited text files because they tend to be full of mistakes and dirty data. Records are missing fields. Fields get swapped with each other. A field that is supposed to contain a number between 1 and 12 may contain a list of foodstuffs the data entry clerk was supposed to buy on his way home one day. All of these things can and do happen, and you have to account for them when doing anything with such data, whether that’s converting it to XML or summarizing it for an annual report.

When you’re tasked with converting legacy data to XML, you just have to roll up your sleeves and attack the problem. You need to understand the current structure of the data. You need to write a program that reads the input format and writes out XML. You need to debug the inevitable problems that arise when the data in the input isn’t exactly you thought it was or what it’s supposed to be. By far the hardest part of this problem is parsing the input data, in whatever form it takes. Once you’ve loaded the data into your program, writing it back out again in XML is a cake walk.

The Budget

As an example of this process, I’m going to use U.S. Federal Government budget authorization data, which the Office of Management and Budget publishes in a variety of equivalent flat formats, even though the data itself is relatively unflat. This is a good example of the sort of legacy data developers often have to deal with. The complete document consists of 3185 line items. Each line item has 43 separate fields. In the comma separated values (CSV) version of the file, a typical line item looks like this:

"418","National Endowment for the Humanities","00","National Endowment for the Humanities","0200","National Endowment for the Humanities: grants and administration","59","503","Research and general education aids","Discretionary","On-budget", 0, 0, 0, 121275, 145231, 150100, 151299, 130560, 135447, 140118, 139478, 132582, 138890, 140435, 153000, 156910, 170002, 175955, 177413, 177491, 172000, 110000, 110000, 111000, 112000, 115000, 120000, 121000, 124000, 126000, 129000, 132000

Each field is separated from the following field by a comma. Strings are enclosed in double quotes, and may contain commas that do not delimit fields. Dollar amounts are written as integers divided by 1000. That is, the last value in the above line is 132 million dollars, not 132 thousand dollars. Table 4.1 identiifes the 43 separate fields.[1]

Table 4.1. Budget Fields

Field number | Field Name | Description
1 | Agency code | A unique 3-digit numerical code for the cabinet department or independent agency
2 | Agency name | The name of the cabinet level department (e.g. Department of Defense) or independent agency (e.g. Peace Corps). Even though only the executive branch actually has true agencies, for purposes of the budget, offices within the legislative branch are given the agency name “Legislative Branch” and offices within the judicial branch are given the agency name “Judicial Branch”. Agency names have a maximum of 89 characters
3 | Bureau code | 2-digit numerical code for the bureau within the cabinet department or independent agency; bureau codes are unique only within an agency.
4 | Bureau name | The name of the sub-office within the agency, e.g. “Coast Guard” or “Federal Aviation Administration”. Amounts for the agency as a whole that are not part of a specific bureau are generally categorized under a fictional bureau with the same name as the agency. Bureau names have a maximum of 89 characters
5 | Account code | A 4-digit code (outlays) or 6-digit code (offsetting receipts) for the account within the bureau.
6 | Account name | The name of the budgeted function within the bureau. Account names have a maximum of 160 characters
7 | Treasury Agency code | 2-digit numerical code for the agency, assigned by the Treasury Department
8 | Subfunction code | 3-digit numerical code for the subfunction within an account
9 | Subfunction title | The name of the subfunction within the account. Subfunction titles have a maximum of 72 characters
10 | BEA category | Budget Enforcement Act category: “Mandatory”, “Discretionary”, or “Net interest”
11 | On- and off-budget indicator | “On-budget” or “Off-budget”; Social Security trust funds and the Postal Service are off-budget, all other accounts are on-budget
12 | 1976 value | Actual amounts, in thousands of dollars, for fiscal year (FY) 1976. Budget authority is usually shown as a positive value. Offsetting receipts are usually negative values.
13 | TQ value | Actual amount, in thousands of dollars, for the “transitional quarter” in 1976 when the government shifted the start of its fiscal year ahead from July to October.
14-37 | 1977-2000 | Actual amounts, in thousands of dollars, for each fiscal year from 1977 to 2000
38-end | 2001-2006 | Estimated amounts, in thousands of dollars, for FY 2001 through FY 2006

[1] Adapted from PUBLIC BUDGET DATABASE USER’s GUIDE: Budget of the United States Government Fiscal Year 2002, Budget Analysis Branch Office of Management and Budget, April 9, 2001, p. 8

Prev | Up | Next
Summary | Home | The Model
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 10, 2001


The Model
Prev Chapter 4. Converting Flat Files to XML Next

The Model

Now that we’ve inspected the existing format, the next step is to design the Java data structures that will hold this information inside the program after it is parsed. Some times you’ll want to design custom Java classes that closely represent the data. If I were writing a budget analysis program, I might do that here. However, since we’re not planning on doing anything more complex than writing the data back out again as XML, the generic data structures in the Java Collections API will more than suffice.

There are an indefinite number of records from year to year as new budget items are added. Thus the list of records will be kept in a java.util.ArrayList. Any other form of java.util.List such as a Vector or a LinkedList would work equally well. After initial construction, I’ll only access this object through the methods of the abstract List superclass. The program will not depend on any implementation details of the list.

The records themselves can be represented as arrays, vectors, instances of a custom class, hashtables or maps. If the data is reasonably clean, I find it easier to use a custom class or a map. An array or vector works well when there may be extra data in some lines or perhaps missing information. In my initial experiments, the data proved to be fairly clean so I chose to use a Map. The keys will be reasonable approximations to the field names so they can be stored in a static array for easy extraction and iteration in a later part of the code. Again, there are no API calls that set this up for you. You have to do it yourself.

Figure 4.1. The list of maps data structure for the budget

When complete, you’ll have a list of maps, one map for each record, as diagrammed in Figure 4.1. This is very close to the form of the input data. It will still need to be manipulated before it’s in the form for the output data. Some manipulations may be straightforward. For instance, it’s very easy to extract all the data for 1982. Just iterate through the list and pull out only the fields that are relevant to 1982 from each map. Other manipulations are more complex. For example, if you want to convert this into a hierarchical structure in which each bureau is part of its agency, you may need to use a sorted data structure or make multiple passes through the list. Maybe you want to reorganize the data by calendar year instead of fiscal year. And perhaps beyond merely reorganizing, you want to perform some calculations on the data, such as summing the total budget for each agency each year. Whatever output you want, it’s just a matter of writing the code to generate it. Once the input data has been parsed, it’s easy to write it out as XML.

Prev | Up | Next
Chapter 4. Converting Flat Files to XML | Home | Input
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 24, 2002


Input
Prev Chapter 4. Converting Flat Files to XML Next

Input

By far the hardest part of this or any similar problem is parsing the non-XML input data. Everything else pales by comparison. Unlike parsing XML, you generally cannot rely on a library to do the hard work for you. You have to do it yourself. And also unlike XML, there’s little guarantee that the data is well-formed. It’s more likely than not that you will encounter incorrectly formatted data.

In this case, since the records are separated into lines, I’ll read each line, one at a time, using the readLine() method of java.io.BufferedReader. This method works well enough as long as the data is in a file, though it’s potentially buggy when the data is served over a network socket.

Each line is dissected into its component fields inside the splitLine() method. Each record is stored in its own map. The keys for the map are read from a constant array, because the fields are always in the same positions in each record.

Caution

For parsing the data out of each line, a lot of Java programmer’s immediately reach for the java.util.StringTokenizer or java.io.StreamTokenizer classes. Don’t. These classes are very strangely designed and rarely do what developers expect them to do. For example, if StreamTokenizer encounters a \n inside a string literal, it will convert it to a linefeed. This makes sense when parsing Java source code, but in most other environments \n are just another two characters with no special meaning. Java’s tokenizer classes are designed for and suited to parsing Java source code. They are not suitable for reading tab-delimited or comma delimited data. If you want to design your program around a tokenization function, you should write one yourself that behaves appropriately for your data format.

Example 4.1 shows the input code. To use this, open an input stream to the file containing the budget data and pass that stream as an argument to the parse() method. You’ll get back a List containing the parsed data. Each object in this list is a Map containing the data for one line item. Both keys and values for this map are strings. Since the keys are constant, they’re stored in a final static array named keys. At various times I plan to use the keys as XML element names, XML attribute names, or SQL field names. Therefore, it’s necessary to begin them all with letters. Thus the keys for the fiscal year fields will be named, FY1976, FY1977, FY1978, and so forth instead of just 1976, 1977, 1978, and so forth. This will mean we won’t be trivially able to store the keys as ints. However, this turns out not have been the case anyway because one of the year fields turns out to be the transitional quarter in 1976. which does not represent a full year and does not have a numeric name.

Caution

In 1976 the government’s fiscal year shifted forward one quarter. That means that the 1977 fiscal year started in October, a quarter after the 1976 fiscal year ended. There was a transitional quarter from July through September that year. Thus not all the data actually represents a whole year. Here, the special case is very much a result of the data itself. Thus the data can’t be fixed. However, it still requires extra code, and makes the examples less clean than they otherwise would be.

This sort of funky data (a year with only three months in it that can easily be confused with another year) is exactly the sort of thing you have to watch out for when processing legacy data. The real world does not always fit into neatly typed categories. There's almost always some outlier data that just doesn't fit the schema. All too often it's been forced into the existing system by some manager or data entry clerk in ways the original designers never intended. This happens all the time. You cannot assume the data actually adheres to its schema, either implicit or explicit.

The code to parse each line of input is hidden inside the private splitLine() method. This code is relatively complex. It iterates through the record looking for comma delimiter characters. However, it has to ignore characters that appear inside quoted strings. Furthermore it must also recognize that the end of the string delimits the last token. Even so, this method is not very robust. It will throw an uncaught exception if any quotes are omitted, or if there are too few fields. It will not notice and report the error if a record contains too many fields.

Example 4.1. A class that parses comma separated values into a List of HashMaps

import java.io.*;
import java.util.*;

public class BudgetData {

  public static List parse(InputStream src) throws IOException {

    // The document as published by the OMB is encoded in Latin-1
    InputStreamReader isr = new InputStreamReader(src, "8859_1");
    BufferedReader in = new BufferedReader(isr);
    List records = new ArrayList();
    String lineItem;
    while ((lineItem = in.readLine()) != null) {
      records.add(splitLine(lineItem));
    }
    return records;

  }

  // the field names in order
  public final static String[] keys = {
    "AgencyCode",
    "AgencyName",
    "BureauCode",
    "BureauName",
    "AccountCode",
    "AccountName",
    "TreasuryAgencyCode",
    "SubfunctionCode",
    "SubfunctionTitle",
    "BEACategory",
    "On-Off-BudgetIndicator",
    "FY1976", "TransitionQuarter", "FY1977", "FY1978", "FY1979",
    "FY1980", "FY1981", "FY1982", "FY1983", "FY1984", "FY1985",
    "FY1986", "FY1987", "FY1988", "FY1989", "FY1990", "FY1991",
    "FY1992", "FY1993", "FY1994", "FY1995", "FY1996", "FY1997",
    "FY1998", "FY1999", "FY2000", "FY2001", "FY2002", "FY2003",
    "FY2004", "FY2005", "FY2006"
   };

  private static Map splitLine(String record) {

    record = record.trim();

    int index = 0;
    Map result = new HashMap();
    for (int i = 0; i < keys.length; i++) {
      //find the next comma
      StringBuffer sb = new StringBuffer();
      char c;
      boolean inString = false;
      while (true) {
        c = record.charAt(index);
        if (!inString && c == '"') inString = true;
        else if (inString && c == '"') inString = false;
        else if (!inString && c == ',') break;
        else sb.append(c);
        index++;
        if (index == record.length()) break;
      }
      String s = sb.toString().trim();
      result.put(keys[i], s);
      index++;
    }

    return result;

  }

}

Prev | Up | Next
The Model | Home | Determining the Output Format
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified August 21, 2001


Determining the Output Format
Prev Chapter 4. Converting Flat Files to XML Next

Determining the Output Format

Once all the data has been stored in a convenient data structure, you have to write it out as XML. The first step in this process is deciding what the XML should look like. XML does offer the possibility of a much more hierarchical structure than is available in the flat format. For instance, you could sort budgets by department and year. You might use the natural hierarchy of the government in which agencies contains bureaus, bureaus contain accounts, and accounts contain subfunctions. I’ll demonstrate several possibilities, beginning with the simplest, a fairly flat XML representation that stays very close to the original flat format.

As published the data is essentially one table. Thus the simplest output format merely duplicates this table structure. This table will be the root element, here called Budget. Each record in the table—that is, each line in the text file—will be encoded as a separate LineItem element. Each field will be encoded in a child element of the LineItem element whose name is the name of that field. Example 4.2 produces this data.

Example 4.2. Naively reproducing the original table structure in XML

import java.io.*;
import java.util.*;

public class FlatXMLBudget {

  public static void convert(List data, OutputStream out)
   throws IOException {

    Writer wout = new OutputStreamWriter(out, "UTF8");
    wout.write("<?xml version=\"1.0\"?>\r\n");
    wout.write("<Budget>\r\n");

    Iterator records = data.iterator();
    while (records.hasNext()) {
      wout.write("  <LineItem>\r\n");
      Map record = (Map) records.next();
      Set fields = record.entrySet();
      Iterator entries = fields.iterator();
      while (entries.hasNext()) {
        Map.Entry entry = (Map.Entry) entries.next();
        String name = (String) entry.getKey();
        String value = (String) entry.getValue();
        // some of the values contain ampersands and less than
        // signs that must be escaped
        value = escapeText(value);

        wout.write("    <" + name + ">");
        wout.write(value);
        wout.write("</" + name + ">\r\n");
      }
      wout.write("  </LineItem>\r\n");
    }
    wout.write("</Budget>\r\n");
    wout.flush();

  }

  public static String escapeText(String s) {

    if (s.indexOf('&') != -1 || s.indexOf('<') != -1
     || s.indexOf('>') != -1) {
      StringBuffer result = new StringBuffer(s.length() + 4);
      for (int i = 0; i < s.length(); i++) {
        char c = s.charAt(i);
        if (c == '&') result.append("&amp;");
        else if (c == '<') result.append("&lt;");
        else if (c == '>') result.append("&gt;");
        else result.append(c);
      }
      return result.toString();
    }
    else {
      return s;
    }

  }

  public static void main(String[] args) {

    try {

      if (args.length < 1) {
       System.out.println("Usage: FlatXMLBudget infile outfile");
       return;
      }

      InputStream in = new FileInputStream(args[0]);
      OutputStream out;
      if (args.length < 2) {
        out = System.out;
      }
      else {
        out = new FileOutputStream(args[1]);
      }

      List results = BudgetData.parse(in);
      convert(results, out);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

}

The main() method reads the name of an input and output file from the command line. It parses the input file using the previously designed BudgetData.parse() method. This produces a List of line items. This list is passed to the convert() method along with an OutputStream the output will be written to. I used an OutputStream rather than a Writer here because with a Writer it’s not possible to pick your encoding, whereas with an OutputStream you can choose the encoding when you chain the Writer to it.

The convert() method iterates through that list. It extracts each record in turn, outputs a <LineItem> start-tag, then iterates through the Map representing the record, outputting each record in turn. The keys in the Map serve double duty, also becoming the names of the child elements of LineItem. The escapeText() method turns any <, >, or & characters that appear in the value into their respective escape sequences. Finally, the </LineItem> end-tag is output.

To some extent I’ve exposed more of the details here than is ideal. This group of classes is not as well encapsulated as I’d normally prefer. For the most part that’s because I’m deliberately trying to show you how everything works internally. I’m not viewing this as reusable code. If I did need to make this more reusable, I’d probably define a Budget class that contained a list of LineItem objects. Public methods in these classes would hide the detailed storage as a List of Maps.

Here’s the prolog, root element start-tag, and first record from the XML output:

<?xml version="1.0"?>
<Budget>
  <LineItem>
    <FY1994>0</FY1994>
    <FY1993>0</FY1993>
    <FY1992>0</FY1992>
    <FY1991>0</FY1991>
    <FY1990>0</FY1990>
    <AccountCode></AccountCode>
    <On-Off-BudgetIndicator>On-budget</On-Off-BudgetIndicator>
    <FY1989>0</FY1989>
   <AccountName>Receipts, Central fiscal operations</AccountName>
    <FY1988>0</FY1988>
    <FY1987>0</FY1987>
    <FY1986>0</FY1986>
    <FY1985>0</FY1985>
    <FY1984>0</FY1984>
    <FY1983>0</FY1983>
    <FY1982>0</FY1982>
    <SubfunctionCode>803</SubfunctionCode>
    <FY1981>0</FY1981>
    <FY2006>0</FY2006>
    <FY1980>0</FY1980>
    <FY2005>0</FY2005>
    <FY2004>0</FY2004>
    <FY2003>0</FY2003>
    <FY2002>0</FY2002>
    <FY2001>0</FY2001>
    <FY2000>0</FY2000>
    <AgencyCode>001</AgencyCode>
    <BEACategory>Mandatory</BEACategory>
    <TransitionQuarter>-132</TransitionQuarter>
    <FY1979>-726</FY1979>
    <FY1978>-385</FY1978>
    <FY1977>-429</FY1977>
    <FY1976>-287</FY1976>
    <TreasuryAgencyCode></TreasuryAgencyCode>
    <AgencyName>Legislative Branch</AgencyName>
    <BureauCode>00</BureauCode>
    <BureauName>Legislative Branch</BureauName>
    <FY1999>0</FY1999>
    <FY1998>0</FY1998>
    <FY1997>0</FY1997>
    <FY1996>0</FY1996>
   <SubfunctionTitle>Central fiscal operations</SubfunctionTitle>
    <FY1995>0</FY1995>
  </LineItem>

As you can see, one thing lost in the transition to XML is the order of the input fields. This is a side effect of storing each record as an unordered HashMap. However, order is really not important for this use-case so nothing significant has been lost. Order is rarely a problem for data-oriented XML applications. If order is important in your data, you can maintain it by using a list or an array to hold the fields instead of a map. You could even use a SortedMap with a custom implementation of the Comparable interface for the keys you store in the map.

The architecture used here is somewhat memory intensive. The entire input data is read in and parsed before the first line is written out. Since in this case, the size of the input data is reaosnably small for modern systems (just over a megabyte), this isn’t really an issue. However, if the data were much bigger it would make sense to stream it in reasonably sized chunks. Each line item could be output as soon as it was read rather than waiting for the last one to be read. For example, instead of storing each record in a big list, the BudgetData.parse() method could immediately write the data onto a Writer:

  public static void parse(InputStream src, OutputStream out)
   throws IOException {

    Writer wout = new OutputStreamWriter(out, "UTF8");
    wout.write("<?xml version=\"1.0\"?>\r\n");
    wout.write("<Budget>\r\n");

    // The document as published by the OMB is encoded in Latin-1
    InputStreamReader isr = new InputStreamReader(src, "8859_1");
    BufferedReader in = new BufferedReader(isr);
    String lineItem;
    while ((lineItem = in.readLine()) != null) {
      Map data = splitLine(lineItem);
      Iterator records = data.iterator();
      wout.write("  <LineItem>\r\n");
      Set fields = records.entrySet();
      Iterator entries = fields.iterator();
      while (entries.hasNext()) {
        Map.Entry entry = (Map.Entry) entries.next();
        String name = (String) entry.getKey();
        String value = (String) entry.getValue();
        // some of the values contain ampersands and less than
        // signs that must be escaped
        value = escapeText(value);

        wout.write("    <" + name + ">");
        wout.write(value);
        wout.write("</" + name + ">\r\n");
      }  // end entries loop
      wout.write("  </LineItem>\r\n");
    }  // end lineitem loop

    wout.write("</Budget>\r\n");
    wout.flush();

  }

The disadvantage to this approach is that it only works when the output is fairly close to the input. In this case, each line of text in the input produces a single element in the output. The other extreme would be reordering the line items by year, essentially exchanging the rows and the columns. In this case, each element of output would include some content from every line of text. We'll see some examples of this (and some in-between cases) shortly. In these cases, a streaming architecture would require multiple passes over the input data and be very inefficient. Still, if the data set were large enough, that might be the only workable approach.

Validation

Error checking in this program is minimal. For instance, the convert() method does not check to see that each item in the List is in fact a Map. If somebody passes in a List that contains non-Map objects, an uncaught ClassCastException will be thrown. convert() also doesn’t check that each Map has all 43 required entries or that the keys in the Map match the known key list. No exception will be thrown in these cases. However, if a record turns out to be short a few fields or to have the wrong fields, then validation of the output should pick it up.

The first test you should make of the output is for simple well-formedness. Whatever tool you normally use to check well-formedness, for example, sax.Counter from Xerces-J, will suffice. The first time I ran the output from this program through sax.Counter, it told me the output was indeed not well-formed. The problem was that some of the names and titles used & characters that had to be escaped, so I added the escapeText() method to the FlatXMLBudget class, and reran the program. I checked my output again, and it was still malformed. Although I’d written the escapeText() method I’d forgotten to invoke it. I edited FlatXMLBudget one more time and reran the program. My output was still malformed. This time I’d forgotten to recompile the program before rerunning it, so I recompiled and ran it one more time. This time my output proved well-formed. This is just normal back-and-forth development. Although the test is specific to XML, there’s nothing XML-specific about the process. You keep fixing problems, and trying again until you succeed. Many fixes will simply expose new bugs and errors. Eventually you fix the last one, and the program runs correctly.

Once you’re confident your program is outputting well-formed XML, the next step is to validate it. Even if your use-case doesn’t actually require valid XML, validation is still an invaluable testing and debugging tool. Example 4.3 is a schema for the intended output of Example 4.2. I chose to use a schema rather than a DTD so I could check that each field contained the correct form of data.

Example 4.3. A schema for the XML budget data

<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  <xsd:element name="Budget">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="LineItem" maxOccurs="unbounded">
          <xsd:complexType>
            <xsd:all>
              <xsd:element name="AccountCode">
                <xsd:simpleType>
                  <xsd:union
                       memberTypes="FourDigitCode SixDigitCode"/>
                </xsd:simpleType>
              </xsd:element>
              <xsd:element name="AccountName">
                <xsd:simpleType>
                  <xsd:restriction base="xsd:string">
                    <xsd:maxLength value="160"/>
                  </xsd:restriction>
                </xsd:simpleType>
              </xsd:element>
              <xsd:element name="AgencyCode"
                           type="ThreeDigitCode"/>
              <xsd:element name="TreasuryAgencyCode"
                           type="TwoDigitCode"/>
              <xsd:element name="AgencyName">
                <xsd:simpleType>
                  <xsd:restriction base="xsd:string">
                    <xsd:maxLength value="89"/>
                  </xsd:restriction>
                </xsd:simpleType>
              </xsd:element>
              <xsd:element name="BEACategory">
                <xsd:simpleType>
                  <xsd:restriction base="xsd:string">
                    <xsd:enumeration value="Mandatory"/>
                    <xsd:enumeration value="Discretionary"/>
                    <xsd:enumeration value="Net interest"/>
                  </xsd:restriction>
                </xsd:simpleType>
              </xsd:element>
              <xsd:element name="BureauCode"
                           type="TwoDigitCode"/>
              <xsd:element name="BureauName">
                <xsd:simpleType>
                  <xsd:restriction base="xsd:string">
                    <xsd:maxLength value="89"/>
                  </xsd:restriction>
                </xsd:simpleType>
              </xsd:element>
              <xsd:element name="On-Off-BudgetIndicator">
                <xsd:simpleType>
                  <xsd:restriction base="xsd:string">
                    <xsd:enumeration value="On-budget"/>
                    <xsd:enumeration value="Off-budget"/>
                  </xsd:restriction>
                </xsd:simpleType>
              </xsd:element>
              <xsd:element name="SubfunctionCode"
                           type="ThreeDigitCode"/>
              <xsd:element name="SubfunctionTitle">
                <xsd:simpleType>
                  <xsd:restriction base="xsd:string">
                    <xsd:maxLength value="72"/>
                  </xsd:restriction>
                </xsd:simpleType>
              </xsd:element>
              <xsd:element name="FY1976" type="xsd:integer"/>
              <xsd:element name="TransitionQuarter"
                           type="xsd:integer"/>
              <xsd:element name="FY1977" type="xsd:integer"/>
              <xsd:element name="FY1978" type="xsd:integer"/>
              <xsd:element name="FY1979" type="xsd:integer"/>
              <xsd:element name="FY1980" type="xsd:integer"/>
              <xsd:element name="FY1981" type="xsd:integer"/>
              <xsd:element name="FY1982" type="xsd:integer"/>
              <xsd:element name="FY1983" type="xsd:integer"/>
              <xsd:element name="FY1984" type="xsd:integer"/>
              <xsd:element name="FY1985" type="xsd:integer"/>
              <xsd:element name="FY1986" type="xsd:integer"/>
              <xsd:element name="FY1987" type="xsd:integer"/>
              <xsd:element name="FY1988" type="xsd:integer"/>
              <xsd:element name="FY1989" type="xsd:integer"/>
              <xsd:element name="FY1990" type="xsd:integer"/>
              <xsd:element name="FY1991" type="xsd:integer"/>
              <xsd:element name="FY1992" type="xsd:integer"/>
              <xsd:element name="FY1993" type="xsd:integer"/>
              <xsd:element name="FY1994" type="xsd:integer"/>
              <xsd:element name="FY1995" type="xsd:integer"/>
              <xsd:element name="FY1996" type="xsd:integer"/>
              <xsd:element name="FY1997" type="xsd:integer"/>
              <xsd:element name="FY1998" type="xsd:integer"/>
              <xsd:element name="FY1999" type="xsd:integer"/>
              <xsd:element name="FY2000" type="xsd:integer"/>
              <xsd:element name="FY2001" type="xsd:integer"/>
              <xsd:element name="FY2002" type="xsd:integer"/>
              <xsd:element name="FY2003" type="xsd:integer"/>
              <xsd:element name="FY2004" type="xsd:integer"/>
              <xsd:element name="FY2005" type="xsd:integer"/>
              <xsd:element name="FY2006" type="xsd:integer"/>
            </xsd:all>
          </xsd:complexType>
         </xsd:element>
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>

  <xsd:simpleType name="TwoDigitCode">
    <xsd:restriction base="xsd:string">
      <xsd:pattern value="[0-9][0-9]"/>
    </xsd:restriction>
  </xsd:simpleType>

  <xsd:simpleType name="ThreeDigitCode">
    <xsd:restriction base="xsd:string">
      <xsd:pattern value="[0-9][0-9][0-9]"/>
    </xsd:restriction>
  </xsd:simpleType>

  <xsd:simpleType name="FourDigitCode">
    <xsd:restriction base="xsd:string">
      <xsd:pattern value="[0-9][0-9][0-9][0-9]"/>
    </xsd:restriction>
  </xsd:simpleType>

  <xsd:simpleType name="SixDigitCode">
    <xsd:restriction base="xsd:string">
      <xsd:pattern value="[0-9][0-9][0-9][0-9][0-9][0-9]"/>
    </xsd:restriction>
  </xsd:simpleType>

</xsd:schema>

As well as indicating bugs in your code that you need to fix, schema checking can uncover dirty input data that isn’t what it’s supposed to be. This happens all the time in real-world data. When I validated the XML output against this schema, a few problems did crop up. A few account codes were blank, and the third BEA category was actually “Net interest”, not “Net Interest” as the documentation claimed. However, these problems were quite minor overall. In fact, I was quite surprised that there weren’t more problems. The programmers at the OMB are doing a much better than average job.

When faced with dirty data that's static, the normal solution is to edit the input data by hand and rerun the program. However if the data is dynamic (e.g. the sales figures for various retail locations submitted to a central server when each franchise closes down for the night) then you’ll have to write extra code to massage the bad data into an acceptable format, at least until you can fix the process that’s generating the dirty data.

Attributes

The dual fact that the order of child elements really isn’t significant within a LineItem and that no two child elements are repeated, suggests that some of this data could be stored as attributes. Besides the obvious change in the convert() method, this also requires changing the escapeText() method to escape any embedded quotes, which are legal in element content but not attribute values. Example 4.4 demonstrates.

Example 4.4. Converting to XML with attributes

import java.io.*;
import java.util.*;

public class AttributesXMLBudget {

  public static void convert(List data, OutputStream out)
   throws IOException {

    Writer wout = new OutputStreamWriter(out, "UTF8");
    wout.write("<?xml version=\"1.0\"?>\r\n");
    wout.write("<Budget>\r\n");

    Iterator records = data.iterator();
    while (records.hasNext()) {
      wout.write("  <ListItem");
      Map record = (Map) records.next();

      // write the attributes
      writeAttribute(wout, "AgencyCode", record);
      writeAttribute(wout, "AgencyName", record);
      writeAttribute(wout, "BureauCode", record);
      writeAttribute(wout, "BureauName", record);
      writeAttribute(wout, "AccountCode", record);
      writeAttribute(wout, "AccountName", record);
      writeAttribute(wout, "TreasuryAgencyCode", record);
      writeAttribute(wout, "SubfunctionCode", record);
      writeAttribute(wout, "SubfunctionTitle", record);
      writeAttribute(wout, "BEACategory", record);
      writeAttribute(wout, "On-Off-BudgetIndicator", record);
      wout.write(">\r\n");
      writeAmount(wout, 1976, record);
      wout.write("    <Amount year='TransitionQuarter'>");
      wout.write(
        escapeText((String) record.get("TransitionQuarter"))
      );
      wout.write("</Amount>\r\n");
      for (int year=1977; year <= 2006; year++) {
        writeAmount(wout, year, record);
      }
      wout.write("  </ListItem>\r\n");
    }
    wout.write("</Budget>\r\n");
    wout.flush();

  }

  // Just a couple of private methods to factor out repeated code
  private static void writeAttribute(Writer out, String name,
   Map record) throws IOException {
    out.write(" " + name + "='"
     + escapeText((String) record.get(name)) + "'");
  }

  private static void writeAmount(Writer out, int year,
   Map record) throws IOException {
    out.write("    <Amount year='" + year + "'>");
    out.write(escapeText((String) record.get("FY" + year)));
    out.write("</Amount>\r\n");
  }

  public static String escapeText(String s) {

    if (s.indexOf('&') != -1 || s.indexOf('<') != -1
     || s.indexOf('>') != -1 || s.indexOf('"') != -1
     || s.indexOf('\'') != -1 ) {
      StringBuffer result = new StringBuffer(s.length() + 6);
      for (int i = 0; i < s.length(); i++) {
        char c = s.charAt(i);
        if (c == '&') result.append("&amp;");
        else if (c == '<') result.append("&lt;");
        else if (c == '"') result.append("&quot;");
        else if (c == '\'') result.append("&apos;");
        else if (c == '>') result.append("&gt;");
        else result.append(c);
      }
      return result.toString();
    }
    else {
      return s;
    }

  }

  public static void main(String[] args) {

    try {

      if (args.length < 1) {
        System.out.println(
         "Usage: AttributesXMLBudget infile outfile"
        );
        return;
      }

      InputStream in = new FileInputStream(args[0]);
      OutputStream out;
      if (args.length < 2) {
        out = System.out;
      }
      else {
        out = new FileOutputStream(args[1]);
      }

      List results = BudgetData.parse(in);
      convert(results, out);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

}

In this case, since different fields are treated differently, the program can’t just iterate through the map. It must ask for each field by name and put it where it belongs. This has the advantage of making the output a little more human legible, as this fragment of output demonstrates.

<?xml version="1.0"?>
<Budget>
  <LineItem AgencyCode='001' AgencyName='Legislative Branch' BureauCode='00'
            BureauName='Legislative Branch' AccountCode=''
            AccountName='receipts, Central fiscal operations'
            TreasuryAgencyCode='' SubfunctionCode='803'
            SubfunctionTitle='Central fiscal operations'
            BEACategory='mandatory' On-Off-BudgetIndicator='On-budget'>
    <Amount year='1976'>-287</Amount>
    <Amount year='TransitionalQuarter'>-132</Amount>
    <Amount year='1977'>-429</Amount>
    <Amount year='1978'>-385</Amount>
    <Amount year='1979'>-726</Amount>
    <Amount year='1980'>0</Amount>
    <Amount year='1981'>0</Amount>
    <Amount year='1982'>0</Amount>
    <Amount year='1983'>0</Amount>
    <Amount year='1984'>0</Amount>
    <Amount year='1985'>0</Amount>
    <Amount year='1986'>0</Amount>
    <Amount year='1987'>0</Amount>
    <Amount year='1988'>0</Amount>
    <Amount year='1989'>0</Amount>
    <Amount year='1990'>0</Amount>
    <Amount year='1991'>0</Amount>
    <Amount year='1992'>0</Amount>
    <Amount year='1993'>0</Amount>
    <Amount year='1994'>0</Amount>
    <Amount year='1995'>0</Amount>
    <Amount year='1996'>0</Amount>
    <Amount year='1997'>0</Amount>
    <Amount year='1998'>0</Amount>
    <Amount year='1999'>0</Amount>
    <Amount year='2000'>0</Amount>
    <Amount year='2001'>0</Amount>
    <Amount year='2002'>0</Amount>
    <Amount year='2003'>0</Amount>
    <Amount year='2004'>0</Amount>
    <Amount year='2005'>0</Amount>
    <Amount year='2006'>0</Amount>
  </LineItem>

The attribute version is a little more compact than the element-only alternative. It also has the advantage of keeping the years in chronological order.

Clearly, there are many other ways you might choose to divide the output between elements and attributes. For instance, you might make all the names elements but all the codes attributes. There’s certainly more than one way to do it. The modifications to produce slightly different forms of this output are straightforward.

Prev | Up | Next
Input | Home | Building Hierarchical Structures from Flat Data
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 08, 2002


Building Hierarchical Structures from Flat Data
Prev Chapter 4. Converting Flat Files to XML Next

Building Hierarchical Structures from Flat Data

Sometimes you want to convert only a subset of the available data. For example, you might want to store each year in a separate XML document. With a few tweaks to the basic code, this is not hard to do. Another possibility is to arrange the data hierarchically, with Agency elements containing Bureau elements which contain Account elements which contain Subfunction elements. This is a little more complex, but far from impossible. Or perhaps you want to massage the data while converting it, for instance by changing the amounts of thousands of dollars to amounts of single dollars. Or you can do all three. The hierarchy would be arranged something like Example 4.5:

Example 4.5. A hierarchical arrangement of the budget data

<?xml version="1.0"?>
<Budget year="2001">
  <Agency>
    <Name>Farm Credit System Financial Assistance Corporation</Name>
    <Code>354</Code>
    <Bureau>
      <Name>Farm Credit System Financial Assistance Corporation</Name>
      <Code>00</Code>
      <Account>
        <Name>Financial assistance corporation trust fund</Name>
        <Code>8202</Code>
        <BEACategory>Mandatory</BEACategory>
        <Subfunction>
          <Title>Farm income stabilization</Title>
          <Code>351</Code>
          <Amount>-1000</Amount>
        </Subfunction>
      </Account>
    </Bureau>
  </Agency>
  <Agency>
    <Name>Department of Education</Name>
    <Code>018</Code>
    <Bureau>
      <Name>Office of Elementary and Secondary Education</Name>
      <Code>10</Code>
      <Account>
        <Name>Reading excellence</Name>
        <Code>0011</Code>
        <BEACategory>Discretionary</BEACategory>
        <Subfunction>
          <Title>
            Elementary, secondary, and vocational education
          </Title>
          <Code>501</Code>
          <Amount>286000000</Amount>
        </Subfunction>
      </Account>
      <Account>
        <Name>Indian education</Name>
        <Code>0101</Code>
        <BEACategory>Discretionary</BEACategory>
        <Subfunction>
          <Title>
            Elementary, secondary, and vocational education
          </Title>
          <Code>501</Code>
          <Amount>116000000</Amount>
        </Subfunction>
      </Account>
      <!-- more accounts...-->
    </Bureau>
    <Bureau>
      <Name>
        Office of Bilingual Education and Minority Languages Affairs
      </Name>
      <Code>15</Code>
      <Account>
        <Name>Bilingual and immigrant education</Name>
        <Code>1300</Code>
        <BEACategory>Discretionary</BEACategory>
        <Subfunction>
          <Title>
            Elementary, secondary, and vocational education
          </Title>
          <Code>501</Code>
          <Amount>460000000</Amount>
        </Subfunction>
      </Account>
    </Bureau>
    <!-- more bureaus...-->
  </Agency>
  <!-- many more agencies... -->
</Budget>

There are two basic approaches to imposing hierarchy on the flat structures we’re starting with. One is to build the hierarchy into the Java data structures in memory. Then these data structures can be output in XML quite simply. The alternative is to leave the data structures in memory flat, but try to insert the hierarchy as the output code is being written. The latter approach is not too difficult when the data can be easily sorted, but most of the time I find the former approach to be simpler.

Although all the data could still be stored in lists of lists and lists of maps and maps of lists and so forth, I really do find that at this point the data structures are clearer if we name them; and in Java there really isn’t any difference between a named data structure and a class. Thus we can describe the structure as classes and objects. Informally, the structure looks like this:

class Budget {

  String year;
  List   agencies;

}

class Agency {

  String code;
  String treasuryCode;
  String name;
  List   bureaus;

}

class Bureau {

  String code;
  String name;
  List   accounts;

}

class Account {

  String code;
  String name;
  String BEACategory;
  List   subfunctions;

}

class Subfunction {

  String code;
  String title;
  long   amount;

}

Note

I must admit that the lack of generic types (templates to C++ programmers) hurts a little here. For example, I want to say that an Agency contains a list of Bureaus. However, all I can really say is that an Agency contains a list of objects and that this list happens to be named bureaus. It’s not a huge problem—if it were, I could always define my own custom list classes with the appropriate types—but it does lead to a less natural solution than generic types do. I thought seriously about using generics for this example, but that would limit this code to Java 1.4. Maybe for the second edition.

Of course, we’ll also need a number of methods in these classes. Factory methods will be used instead of constructors to guarantee that 19 IRS Bureau objects aren’t created just because the IRS has 19 line items in the budget. (One IRS is bad enough. :-) ) Each class will have a getXML() method that returns a String containing the XML form of the object. And each class will have a method to add a child node of the appropriate type. Figure 4.2 shows a UML static structure diagram for the complete group of classes.

Figure 4.2. A UML diagram for the budget class hierarchy

The Budget class, Example 4.6 is at the top of the tree. It contains a list of agencies and a year. Line items encoded as maps by BudgetData are fed into the tree through the add() method. This method breaks that map up into the relevant parts, encodes each as an object, adds each of those objects to the tree in the right place, and stores the agency in the list of agencies. The various add() methods in both this and the other classes are responsible for putting each piece in the right place.

Example 4.6. The Budget class

import java.util.*;

public class Budget {

  private List   agencies = new ArrayList();
  private String year;

  public Budget(String year) {
    this.year = year;
  }

  public void add(Agency agency) {
    if (!agencies.contains(agency)) agencies.add(agency);
  }

  public void add(Map lineItem) {

    String agencyName = (String) lineItem.get("AgencyName");
    agencyName = escapeText(agencyName);
    String agencyCode = (String) lineItem.get("AgencyCode");
    String treasuryAgencyCode
     = (String) lineItem.get("TreasuryAgencyCode");
    Agency agency = Agency.getInstance(agencyName, agencyCode,
     treasuryAgencyCode, year);
    this.add(agency);

    String bureauName = (String) lineItem.get("BureauName");
    bureauName = escapeText(bureauName);
    String bureauCode = (String) lineItem.get("BureauCode");
    Bureau bureau = Bureau.getInstance(bureauName, bureauCode,
     agencyCode, year);
    agency.add(bureau);

    // Names and codes of two accounts in different bureaus
    // can be the same
    String accountName = (String) lineItem.get("AccountName");
    accountName = escapeText(accountName);
    String accountCode = (String) lineItem.get("AccountCode");
    String category    = (String) lineItem.get("BEACategory");
    Account account = Account.getInstance(accountName,
     accountCode, category, bureauCode, agencyCode, year);
    bureau.add(account);

    // Names and codes of two subfunctions in different accounts
    // can be the same
    String subfunctionTitle
     = escapeText((String) lineItem.get("SubfunctionTitle"));
    String subfunctionCode
     = (String) lineItem.get("SubfunctionCode");
    String yearKey = year;
    if (!yearKey.equals("TransitionQuarter")) {
      yearKey = "FY" + year;
    }
    long amount
     = 1000L * Long.parseLong((String) lineItem.get(yearKey));
    Subfunction subfunction = new Subfunction(subfunctionTitle,
     subfunctionCode, amount);
    account.add(subfunction);

  }

  public String getXML() {

    StringBuffer result = new StringBuffer("<Budget year='"
     + this.year +"'>\r\n");
    Iterator iterator = agencies.iterator();
    while (iterator.hasNext()) {
      Agency agency = (Agency) iterator.next();
      result.append(agency.getXML());
    }
    result.append("</Budget>\r\n");
    return result.toString();

  }

  public static String escapeText(String s) {

    if (s.indexOf('&') != -1 || s.indexOf('<') != -1
     || s.indexOf('>') != -1 || s.indexOf('"') != -1
     || s.indexOf('\'') != -1 ) {
      StringBuffer result = new StringBuffer(s.length() + 6);
      for (int i = 0; i < s.length(); i++) {
        char c = s.charAt(i);
        if (c == '&') result.append("&amp;");
        else if (c == '<') result.append("&lt;");
        else if (c == '"') result.append("&quot;");
        else if (c == '\'') result.append("&apos;");
        else if (c == '>') result.append("&gt;");
        else result.append(c);
      }
      return result.toString();
    }
    else {
      return s;
    }

  }

}

Most of the other classes are structurally the same with subtle variations. Each has a list of its children, a private constructor, a public factory method, an add() method to add a child of the appropriate type, and a getXML() method that returns the XML representation of the object as a String. I’ll begin with the Agency class in Example 4.7.

Example 4.7. The Agency class

import java.util.*;

public class Agency {

  private String code;
  private String name;
  private String treasuryCode;
  private String year;

  private List   bureaus = new ArrayList();

  private static Map instances = new HashMap();

  // A private constructor so instantiators
  // have to use the factory method
  private Agency(String name, String code, String treasuryCode,
     String year) {

    this.name = name;
    this.code = code;
    this.treasuryCode = treasuryCode;
    this.year = year;

  }

  public static Agency getInstance(String name, String code,
     String treasuryCode, String year) {

    // Agencies can be uniquely identified by code alone
    String key = code+" "+year;
    Agency agency = (Agency) instances.get(key);
    if (agency == null) {
      agency = new Agency(name, code, treasuryCode, year);
      instances.put(key, agency);
    }

    return agency;

  }

  public void add(Bureau b) {
    if (!bureaus.contains(b)) {
        bureaus.add(b);
    }
  }

  public String getXML() {

    StringBuffer result = new StringBuffer("  <Agency>\r\n");
    result.append("    <Name>" + name + "</Name>\r\n");
    result.append("    <Code>" + code + "</Code>\r\n");
    result.append("    <TreasuryAgencyCode>" + treasuryCode
     + "</TreasuryAgencyCode>\r\n");
    Iterator iterator = bureaus.iterator();
    while (iterator.hasNext()) {
      Bureau bureau = (Bureau) iterator.next();
      result.append(bureau.getXML());
    }
    result.append("  </Agency>\r\n");
    return result.toString();

  }

}

This is not a general purpose class. The public interface is limited to a factory method, a method to add a Bureau to the Agency, and a method to get the XML representation of the object as a String. There aren’t even any getter methods. I’ve deliberately chosen not to burden this and the other classes with a lot of error checking. I’m assuming that will be done both in the database before the data is input here and later through validation after the XML document has been produced. Doing it here too just seemed a tad superfluous.

Clients retrieve instances of the class using the getInstance() factory method. The first time this class is asked to get a particular agency it constructs it using the private constructor, stores it in the static instances map, and returns a reference to it. Subsequent requests will return the formerly created object from the map. The key for the map is the concatenation of the agency’s code and year.

The add() method simply stores Bureau objects in the bureaus list. Since bureaus is private and only accessed through this add() method, we can be certain that no non-Bureau objects will ever get added to this list. We can further guarantee that the list doesn’t contain any duplicates.

The getXML() method returns an XML representation of this Agency object. It loops through all the bureaus the agency contains, and uses their getXML() methods to get the XML representations for each of them. The XML is returned as a String. An alternative approach that might be more memory efficient and perhaps a little faster, especially in streaming applications, would be to pass the OutputStream or Writer that the XML should be written onto to the getXML() method. In essence, this would be a writeXML() method rather than a getXML() method.

The code here is far from the most efficient possible. There are a lot of optimizations that could be done, including many that would build far fewer intermediate string representations of each element. For instance, you could pass in the StringBuffer to append to to each getXML() method. However, since this is intended primarily as a one-off solution, the optimization didn’t seem to be worth the investment of time or the added complexity.

The Bureau class shown in Example 4.8 is much the same except that it contains a list of accounts. Furthermore, both an agency code and a bureau code are required to uniquely identify bureaus. The bureau code alone is not sufficient, so the keys for the instances map are formed by concatenating the agency code, the bureau code, and the year.

Example 4.8. The Bureau Class

import java.util.*;

public class Bureau {

  // Agency code plus bureau code uniquely identify a bureau
  // Bureau code alone is definitely not sufficient
  private String code;
  private String name;
  private String year;
  private String agencyCode;

  private List   accounts = new ArrayList();

  private static Map instances = new HashMap();

  // Use a private constructor so instantiators
  // have to use the factory method
  private Bureau(String name, String code, String agencyCode,
    String year) {

    this.name = name;
    this.code = code;
    this.agencyCode = agencyCode;
    this.year = year;

  }

  public static Bureau getInstance(String name, String code,
   String agencyCode, String year) {

    String key = agencyCode+" "+code+" "+year;
    Bureau bureau = (Bureau) instances.get(key);
    if (bureau == null) {
      bureau = new Bureau(name, code, agencyCode, year);
      instances.put(key, bureau);
    }

    return bureau;

  }

  public void add(Account account) {
    if (!accounts.contains(account)) accounts.add(account);
  }

  public String getXML() {

    StringBuffer result = new StringBuffer("    <Bureau>\r\n");
    result.append("      <Name>" + name + "</Name>\r\n");
    result.append("      <Code>" + code + "</Code>\r\n");
    Iterator iterator = accounts.iterator();
    while (iterator.hasNext()) {
      Account account = (Account) iterator.next();
      result.append(account.getXML());
    }
    result.append("    </Bureau>\r\n");
    return result.toString();

  }

}

The Account class shown in Example 4.9 is similar to Bureau and Agency. Accounts are uniquely identified by an account code, a bureau code, an agency code, and a BEA code. Otherwise, this class is structured the same as the last two.

Example 4.9. An Account Class

import java.util.*;

public class Account {

  // An account is uniquely identified by account code,
  // bureau code, agency code and BEA category
  private String code;
  private String name;
  private String BEACategory;
  private String bureauCode;
  private String agencyCode;
  private String year;

  private List   subfunctions = new ArrayList();

  private static Map instances = new HashMap();

  // Use a private constructor so clients
  // have to use the factory method
  private Account(String name, String code, String BEACategory,
   String bureauCode, String agencyCode, String year) {

    this.name = name;
    this.code = code;
    this.BEACategory = BEACategory;
    this.bureauCode = bureauCode;
    this.agencyCode = agencyCode;
    this.year = year;

  }

  public static Account getInstance(String name, String code,
   String BEACategory, String bureauCode, String agencyCode,
   String year) {

    String key = code + " " + BEACategory + " " + bureauCode
     + " " + agencyCode + " " + year;
    Account account = (Account) instances.get(key);
    if (account == null) {
      account = new Account(name, code, BEACategory, bureauCode,
       agencyCode, year);
      instances.put(key, account);
    }

    return account;

  }

  public void add(Subfunction sfx) {
    if (!subfunctions.contains(sfx)) subfunctions.add(sfx);
  }

  public String getXML() {

    StringBuffer result = new StringBuffer();
    result.append("      <Account>\r\n");
    result.append("        <Name>" + name + "</Name>\r\n");
    result.append("        <Code>" + code + "</Code>\r\n");
    result.append("        <BEACategory>" + BEACategory
     + "</BEACategory>\r\n");
    Iterator iterator = subfunctions.iterator();
    while (iterator.hasNext()) {
      Subfunction subfunction = (Subfunction) iterator.next();
      result.append(subfunction.getXML());
    }
    result.append("      </Account>\r\n");
    return result.toString();

  }

}

Finally the Subfunction class, Example 4.10, contains a title, a code, and an amount of money for a given year. Unlike agencies, bureaus, and accounts, subfunctions are not necessarily unique. For example, the “Department of Defense-Military” subfunction appears in 222 different accounts. Therefore this class does not use a factory method, just a regular constructor.

Example 4.10. The Subfunction Class

import java.util.*;

public class Subfunction {

  private String code;
  private String title;
  private long   amount;

  public Subfunction(String title, String code, long amount) {

    this.title  = title;
    this.code   = code;
    this.amount = amount;

  }

  public String getXML() {

    StringBuffer result
      = new StringBuffer("        <Subfunction>\r\n");
    result.append("          <Name>" + title + "</Name>\r\n");
    result.append("          <Code>" + code + "</Code>\r\n");
    result.append("          <Amount>");
    result.append(amount + "</Amount>\r\n");
    result.append("        </Subfunction>\r\n");
    return result.toString();

  }

}

The ultimate plan is to read all the records of the budget data, and use that information to create objects of these classes. As each object is created, it will be added to its parent object’s list. When all records have been processed, the getXML() method of the Budget object will be invoked to retrieve the XML representation of the budget. This XML can be written onto an output stream. Example 4.11 does this. It has a main() method to provide a simple user interface. This just reads the year and the input and output file names from the user, and calls convert(). The convert() method delegates reading the input to the BudgetData class and the data-structure building and most of the output to the Budget class.

Example 4.11. The driver class that builds the data structure and writes it out again

import java.io.*;
import java.util.*;

public class HierarchicalXMLBudget {

  public static void convert(List budgetData, String year,
   OutputStream out) throws IOException {

    Budget budget = new Budget(year);
    Iterator records = budgetData.iterator();
    while (records.hasNext()) {
      Map lineItem = (Map) records.next();
      budget.add(lineItem);
    }

    Writer wout = new OutputStreamWriter(out, "UTF8");
    wout.write("<?xml version=\"1.0\"?>\r\n");
    wout.write(budget.getXML());
    wout.flush();

  }

  public static void main(String[] args) {

    try {

      if (args.length < 2) {
        System.out.println(
         "Usage: HierarchicalXMLBudget year infile outfile");
        return;
      }

      // simple error checking on the year value
      try {
        if (!args[0].equals("TransitionQuarter")) {
          Integer.parseInt(args[0]);
        }
      }
      catch (NumberFormatException e) {
        System.out.println(
         "Usage: HierarchicalXMLBudget year infile outfile");
        return;
      }

      InputStream in = new FileInputStream(args[1]);
      OutputStream out;
      if (args.length < 3) {
        out = System.out;
      }
      else {
        out = new FileOutputStream(args[2]);
      }

      List results = BudgetData.parse(in);
      convert(results, args[0], out);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

}

Here’s the start of a sample run:

C:\XMLJAVA>java HierarchicalXMLBudget 2002 budauth.txt
<?xml version="1.0"?>
<Budget year='2002'>
  <Agency>
    <Name>Legislative Branch</Name>
    <Code>001</Code>
    <TreasuryAgencyCode></TreasuryAgencyCode>
    <Bureau>
      <Name>Legislative Branch</Name>
      <Code>00</Code>
      <Account>
        <Name>Receipts, Central fiscal operations</Name>
        <Code></Code>
        <BEACategory>Mandatory</BEACategory>
        <Subfunction>
          <Name>Central fiscal operations</Name>
          <Code>803</Code>
          <Amount>0</Amount>
        </Subfunction>
      </Account>
      <Account>
        <Name>Receipts, Central fiscal operations</Name>
        <Code></Code>
        <BEACategory>Net interest</BEACategory>
        <Subfunction>
...

The technique used here is actually an extremely powerful one. The data is read one item at a time. Each item is stored in a data structure. The data structure automatically moves each item into its appropriate slot. The final result is generated by a single method call to the root of the data structure.

Note

A similar approach would be to define classes that represent the XML elements and attributes rather than the input data. Indeed this is the approach taken by the Document Object Model, DOM. I’ll explore this alternative in coming chapters.

Prev | Up | Next
Determining the Output Format | Home | Alternatives to Java
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified August 17, 2001


Alternatives to Java
Prev Chapter 4. Converting Flat Files to XML Next

Alternatives to Java

When all you have is a hammer, most problems look a lot like nails. Since you’re reading this book, I’m willing to bet that Java is your hammer of choice, and indeed Java is a very powerful hammer. However, sometimes you really could use a screwdriver; and this may be one of those times. I must admit that the solution for imposing hierarchy developed in the last section feels more than a little like pounding a screw with a hammer. Maybe it would be better to use the hammer to set the screw, but then use a screwdriver to drive it in. In this section I want to explore a few possible screwdrivers including XSLT and XQuery. Rather than using such complex Java code, I’ll use Java to get the data into the simple XML format produced by Example 4.2 that closely matches the flat input data. Then I’ll use XSLT to transform this simple intermediate XML format into the less-flat final XML format. To refresh your memory, the flat XML data is organized like this:

<?xml version="1.0"?>
<Budget>
  <LineItem>
    <FY1994>-1982</FY1994>
    <FY1993>4946</FY1993>
    <FY1992>-3251</FY1992>
    <FY1991>-17373</FY1991>
    <FY1990>-90008</FY1990>
    <AccountCode>265197</AccountCode>
    <On-Off-BudgetIndicator>On-budget</On-Off-BudgetIndicator>
    <TransitionQuarter>0</TransitionQuarter>
    <FY1989>-80069</FY1989>
    <AccountName>Sale of scrap and salvage materials</AccountName>
    <FY1988>-72411</FY1988>
    <FY1987>-60964</FY1987>
    <FY1986>-61462</FY1986>
    <FY1985>-68182</FY1985>
    <FY1984>-79482</FY1984>
    <FY1983>0</FY1983>
    <FY1982>0</FY1982>
    <SubfunctionCode>051</SubfunctionCode>
    <FY1981>0</FY1981>
    <FY2006>-1000</FY2006>
    <FY1980>0</FY1980>
    <FY2005>-1000</FY2005>
    <FY2004>-1000</FY2004>
    <FY2003>-1000</FY2003>
    <FY2002>-1000</FY2002>
    <FY2001>-1000</FY2001>
    <FY2000>-2000</FY2000>
    <AgencyCode>007</AgencyCode>
    <BEACategory>Mandatory</BEACategory>
    <FY1979>0</FY1979>
    <FY1978>0</FY1978>
    <FY1977>0</FY1977>
    <FY1976>0</FY1976>
    <TreasuryAgencyCode>97</TreasuryAgencyCode>
    <AgencyName>Department of Defense--Military</AgencyName>
    <BureauCode>00</BureauCode>
    <BureauName>Department of Defense--Military</BureauName>
    <FY1999>-1000</FY1999>
    <FY1998>-2000</FY1998>
    <FY1997>-4000</FY1997>
    <FY1996>-1000</FY1996>
    <SubfunctionTitle>Department of Defense-Military</SubfunctionTitle>
    <FY1995>-1000</FY1995>
  </LineItem>
  <!-- several thousand more LineItem elements... -->
</Budget>

Imposing Hierarchy with XSLT

The XSLT stylesheet shown in Example 4.12 will convert flat XML budget data of this type into an output document of the same form produced by Example 4.11. Because the input file is so large, you may need to raise the memory allocation for your XSLT processor before running the transform.

Example 4.12. An XSLT stylesheet that converts flat XML data to hierarchical XML data

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <!-- Try to make the output look half decent -->
  <xsl:output indent="yes" encoding="ISO-8859-1"/>

  <!-- Muenchian method -->
  <xsl:key name="agencies" match="LineItem" use="AgencyCode"/>
  <xsl:key name="bureaus"  match="LineItem"
    use="concat(AgencyCode,'+',BureauCode)"/>
  <xsl:key name="accounts" match="LineItem"
    use="concat(AgencyCode,'+',BureauCode,'+',AccountCode)"/>
  <xsl:key name="subfunctions" match="LineItem"
    use="concat(AgencyCode,'+',BureauCode,'+',AccountCode,
    '+',SubfunctionCode)"/>

  <xsl:template match="Budget">
    <Budget year='2001'>
      <xsl:for-each select="LineItem[generate-id()
       = generate-id(key('agencies',AgencyCode)[1])]">
        <Agency>
          <Name><xsl:value-of select="AgencyName"/></Name>
          <Code><xsl:value-of select="AgencyCode"/></Code>
          <xsl:for-each
            select="/Budget/LineItem[AgencyCode
            =current()/AgencyCode]
             [generate-id() =
               generate-id(key('bureaus',
                     concat(AgencyCode, '+', BureauCode))[1])]">
            <Bureau>
              <Name><xsl:value-of select="BureauName"/></Name>
              <Code><xsl:value-of select="BureauCode"/></Code>
              <xsl:for-each select="/Budget/LineItem
                  [AgencyCode=current()/AgencyCode]
                  [BureauCode=current()/BureauCode]
                  [generate-id() = generate-id(key('accounts',
                   concat(AgencyCode,'+',BureauCode,'+',
                                            AccountCode))[1])]">
                <Account>
                  <Name>
                    <xsl:value-of select="AccountName"/>
                  </Name>
                  <Code>
                    <xsl:value-of select="AccountCode"/>
                  </Code>
                  <xsl:for-each select=
                    "/Budget/LineItem
                     [AgencyCode=current()/AgencyCode]
                     [BureauCode=current()/BureauCode]
                     [AccountCode=current()/AccountCode]
                     [generate-id()=generate-id(
                       key('subfunctions', concat(AgencyCode,'+',
                       BureauCode,'+',AccountCode,'+',
                       SubfunctionCode))[1])]">
                    <Subfunction BEACategory="{BEACategory}"
                     BudgetIndicator="{On-Off-BudgetIndicator}">
                      <Title>
                       <xsl:value-of select="SubfunctionTitle"/>
                      </Title>
                      <Code>
                       <xsl:value-of  select="SubfunctionCode"/>
                      </Code>
                      <Amount>
                        <xsl:value-of select="FY2001"/>
                      </Amount>
                    </Subfunction>
                  </xsl:for-each>
                </Account>
              </xsl:for-each>
            </Bureau>
          </xsl:for-each>
        </Agency>
      </xsl:for-each>
    </Budget>
  </xsl:template>

</xsl:stylesheet>

The algorithm for converting flat data to hierarchical data with XSLT is known as the Muenchian method after its inventor, Steve Muench of Oracle. The trick of the Muenchian method is to use the xsl:key element and the key() function to create node-sets of all the LineItem elements that share the same agency, bureau, account, or subfunction. Inside the template, the generate-id() function is used to compare the current node to the first node in any given group. Output is generated only if we are indeed processing the first Agency, Bureau, Account or Subfunction element with a specified code. Also note, that the select attributes in the xsl:for-each elements keep returning to the root rather than processing children and descendants as is customary. This reflects the fact that the hierarchy in the input is not the same as the hierarchy in the output.

Note

XSLT 2.0 is going to make it much easier to write style sheets that group elements in this fashion. This will likely involve a new xsl:for-each-group element that groups elements according to an XPath expression a current-group() function that selects all the members of the current group so they can be processed together.

One minor advantage of using XSLT instead of Java data structures is that XSLT preserves the order of the input data. You’ll notice that the output begins with the Legislative Branch agency and bureau and the Receipts, Central fiscal operations account, the same as the input data does. This was not the case for the output produced by Java.

<?xml version="1.0" encoding="ISO-8859-1"?>
<Budget year="2001">
   <Agency>
      <Name>Legislative Branch</Name>
      <Code>001</Code>
      <Bureau>
         <Name>Legislative Branch</Name>
         <Code>00</Code>
         <Account>
            <Name>Receipts, Central fiscal operations</Name>
            <Code/>
            <Subfunction BEACategory="Mandatory"
              BudgetIndicator="On-budget">
               <Title>Central fiscal operations</Title>
               <Code>803</Code>
               <Amount>0</Amount>
            </Subfunction>
            <Subfunction BEACategory="Net interest"
              BudgetIndicator="On-budget">
               <Title>Other interest</Title>
               <Code>908</Code>
               <Amount>0</Amount>
            </Subfunction>
         </Account>
         <Account>
            <Name>Charges for services to trust funds</Name>
            ...

The XML Query Language

Caution

This section describes bleeding edge technology. The broad picture presented here is likely to be correct, but the details are almost certain to change. Furthermore, the exact subset of XQuery implemented by early experimental tools varies a lot from one product to the next.

XSLT is Turing complete. Nonetheless some operations are more than a little cumbersome in XSLT. Using the Muenchian method to impose hierarchy is definitely not something envisioned by XSLT’s inventors. The W3C has begun work on a language more suitable for querying XML documents. This language is called, simply enough, the XML Query Language, or XQuery for short. XQuery is to XML documents as SQL is to relational tables. However, XQuery is limited to SELECT. It does not have any equivalent of INSERT, UPDATE, or DELETE. It is a read-only language.

XQuery queries are not in general well-formed XML. Although there is an XML syntax for XQuery, it is not intended to be used by human beings. Instead humans are supposed to write in a more natural 4GL syntax which will be compiled to XML documents if necessary. If you think about it, this shouldn’t be so surprising. SQL statements aren’t tables. Why should XQuery statements be XML documents?

The basic nature of an XQuery query is the FLWR (pronounced “flower”) statement. This is an acronym for for-let-where-return, the basic form of an XQuery query. In brief, for each node in a node-set let a variable have a certain value, where some condition is true and return an XML fragment based on the values of these variables. Variables are set and XML returned using XPath 2.0 expressions.

For example, here’s an XQuery that generates a list of agency names from the flat XML budget:

for $name in document("budauth.xml")/Budget/LineItem/AgencyName
return $name

The for clause iterates over every node in the node set returned by the XPath 2.0 expression document("budauth.xml")/Budget/LineItem/AgencyName. This expression returns a node set containing 3175 AgencyName elements. The XQuery variable $name is set to each of these elements in turn. The return clause is evaluated for each value of $name. In this case, the return clause says to simply return the node the $name variable currently points to. In this example, the $name variable always points to an AgencyName element so the output would begin like this:

<AgencyName>Legislative Branch</AgencyName>
<AgencyName>Legislative Branch</AgencyName>
<AgencyName>Legislative Branch</AgencyName>
<AgencyName>Legislative Branch</AgencyName>
<AgencyName>Legislative Branch</AgencyName>
…

This is not a well-formed XML document because it does not have a root element. However, it is a well-formed XML document fragment.

You can use the XPath 2.0 distinct-values() function around the XPath expression to select only one of each AgencyName elements:

for $name in distinct-values(document("budauth.xml")/Budget/LineItem/AgencyName)
return $name

The output would now begin like this, repeating each agency name only once:

<AgencyName>Legislative Branch</AgencyName>
<AgencyName>Judicial Branch</AgencyName>
<AgencyName>Department of Agriculture</AgencyName>
<AgencyName>Department of Commerce</AgencyName>
<AgencyName>Department of Defense--Military</AgencyName>
…

As well as copying existing elements, XQuery can create new elements. You can type the tags literally where you want them to appear. To include the value of a variable (or other expression) inside the tags, enclose it in curly braces. For example, this query places <Name> and </Name> tags around each agency name rather than <AgencyName> and </AgencyName>. Notice also that it only selects the text content of each AgencyName element rather than the complete element node:

for $name in distinct-values(document("budauth.xml")//AgencyName/text())
return <Name>{ $name }</Name>

The output now begins like this:

<Name>Legislative Branch</Name>
<Name>Judicial Branch</Name>
<Name>Department of Agriculture</Name>
<Name>Department of Commerce</Name>
<Name>Department of Defense--Military</Name>
…

More complex queries typically require multiple variables. These can be set in a let clause based on XPath expressions that refer to the variable in the for clause. For example, this query selects distinct agency codes but returns agency names:

for $code in distinct-values(document("budauth.xml")//AgencyCode)
let $name := $code/../AgencyName
return $name

A where clause can further restrict the members of the node set for which results are generated. where conditions can use boolean connectors such as and, or, and not()). For example, this query finds all the bureaus in the Department of Agriculture:

for $bureau in distinct-values(document("budauth.xml")/Budget/LineItem/BureauName)
where $bureau/../AgencyName = "Department of Agriculture"
return $bureau

XQuery expressions may nest. That is, the return statement of the FLWR may contain another FLWR. For example, this statement lists all the bureau names inside their respective agencies:

for $ac in distinct-values(document("budauth.xml")//AgencyCode)
return
  <Agency>
    <Name>{ $ac/../AgencyName/text() }</Name>
    {
      for $bc in distinct-values(document("budauth.xml")//BureauCode)
      where $bc/../AgencyCode = $ac
      return
        <Bureau>
          <Name>{ $bc/../BureauName/text() }</Name>
        </Bureau>
    }
  </Agency>

The output now begins like this:

<Agency>
  <Name>Legislative Branch</Name>
  <Bureau>
    <Name>Legislative Branch</Name>
  </Bureau>
  <Bureau>
    <Name>Senate</Name>
  </Bureau>
  <Bureau>
    <Name>House of Representatives</Name>
  </Bureau>
  <Bureau>
    <Name>Joint Items</Name>
  </Bureau>
…

This is all the syntax needed to write a query that will convert flat budget data such as that produced by Example 4.2 into a hierarchical XML document. Such a query is shown in Example 4.13 which selects the data from just 2001.

Example 4.13. An XQuery that converts flat data to hierarchical data

<Budget year="2001">
  {
  for $ac in distinct-values(document("budauth.xml")//AgencyCode)
  return
    <Agency>
      <Name>{ $ac/../AgencyName/text() }</Name>
      <Code>{ $ac/text() }</Code>
      {
        for $bc
         in distinct-values(document("budauth.xml")//BureauCode)
        where $bc/../AgencyCode = $ac
        return
          <Bureau>
            <Name>{ $bc/../BureauName/text() }</Name>
            <Code>{ $bc/text() }</Code>
            {
            for $acct in distinct-values(
             document("budauth.xml")//AccountCode)
            where $acct/../AgencyCode = $ac
             AND $acct/../BureauCode = $bc
            return
              <Account
                BEACategory="{ $acct/../BEACategory/text() }">
                <Name>{ $acct/../AccountName/text() }</Name>
                <Code>{ $acct/text() }</Code>
                {
                  for $sfx
                    in document("budauth.xml")//SubfunctionCode
                  where $sfx/../AgencyCode = $ac
                    and $sfx/../BureauCode = $bc
                    and $sfx/../AccountCode = $acct
                  return
                    <Subfunction>
                 <Title>{$sfx/../SubfunctionTitle/text()}</Title>
                      <Code>{ $sfx/text() }</Code>
                      <Amount>{ $sfx/../FY2001/text() }</Amount>
                    </Subfunction>
               }
            </Account>
           }
          </Bureau>
      }
    </Agency>
  }
</Budget>

There’s a lot more to XQuery, but this should give you the idea of what it can do. It’s definitely worth a look any time you have to perform database-like operations on XML documents.

Prev | Up | Next
Building Hierarchical Structures from Flat Data | Home | Relational Databases
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 25, 2002


Relational Databases
Prev Chapter 4. Converting Flat Files to XML Next

Relational Databases

Relational databases are another common source of data for XML documents. Some databases such as FileMaker Pro have built-in support for outputting tables as XML documents. Others do not. However, even if your database can export tables as XML documents, its XML format may not be the XML format you want. Fortunately, as long as there’s a JDBC driver for your database of choice, it’s not hard to extract the information from it and write that information into an XML document in the desired form.

For this example, I’ll use the same budget data previously read out of a CSV file in a single relational table that reflects the original flat structure of the files distributed by the Office of Management and Budget. Doubtless they have the data in their own relational databases, probably divided up into multiple tables; but they don’t publish it that way. They do state that “If you plan to use these data in a relational database, you should designate the following fields as ‘primary’ to uniquely identify each row of data: agency code, bureau code, account code, subfunction code, BEA category, Grant/Nongrant, and On- Off-budget field.” The SQL CREATE TABLE statement that initializes this table is:

CREATE TABLE BudgetAuthorizationTable (
  AgencyCode             CHAR(3),
  AgencyName             VARCHAR(89),
  BureauCode             CHAR(2),
  BureauName             VARCHAR(89),
  AccountCode            VARCHAR(6),
  AccountName            VARCHAR(160),
  TreasuryAgencyCode     CHAR(2),
  SubfunctionCode        CHAR(3),
  SubfunctionTitle       VARCHAR(72),
  BEACategory            VARCHAR(13),
  On-Off-BudgetIndicator VARCHAR(10),
  FY1976                 INTEGER,
  TransitionQuarter      INTEGER,
  FY1977                 INTEGER,
  FY1978                 INTEGER,
  FY1979                 INTEGER,
  FY1980                 INTEGER,
  FY1981                 INTEGER,
  FY1982                 INTEGER,
  FY1983                 INTEGER,
  FY1984                 INTEGER,
  FY1985                 INTEGER,
  FY1986                 INTEGER,
  FY1987                 INTEGER,
  FY1988                 INTEGER,
  FY1989                 INTEGER,
  FY1990                 INTEGER,
  FY1991                 INTEGER,
  FY1992                 INTEGER,
  FY1993                 INTEGER,
  FY1994                 INTEGER,
  FY1995                 INTEGER,
  FY1996                 INTEGER,
  FY1997                 INTEGER,
  FY1998                 INTEGER,
  FY1999                 INTEGER,
  FY2000                 INTEGER,
  FY2001                 INTEGER,
  FY2002                 INTEGER,
  FY2003                 INTEGER,
  FY2004                 INTEGER,
  FY2005                 INTEGER,
  FY2006                 INTEGER,
  PRIMARY KEY (AgencyCode, BureauCode, AccountCode,
               SubfunctionCode, BEACategory, On-Off-BudgetIndicator)
);

The specific database I chose for this example is Microsoft Excel, mostly because it could very easily read the comma delimited files I was starting with. Excel isn’t the best example of a relational database. In fact, it isn’t a relational database at all. However, it does allow you to define a range of cells as a table, and then associate that table with an ODBC data source. This data source can then be read with SQL using JDBC through the JdbcOdbcDriver, which is all I really want to show here. Aside from the choice of JDBC driver, all statements will be completely database independent. The name of the ODBC data source is budauth. The name of the table in that source is BudgetAuthorizationTable.

Extracting the necessary data from the database is just a matter of SQL which, as a Java programmer, you access via JDBC. To some extent you can let the database do the hard work for you by executing the right sequence of SQL commands. In this case, this is just a long sequence of nested SELECT statements. SQL’s DISTINCT operator will be particularly helpful. The contortions of the Muenchian method in Example 4.12 were a roundabout way of providing a distinct operation in XSLT.

Example 4.14. A program that connects to a relational database using JDBC and converts the table to hierarchical XML

import java.sql.*;
import java.io.*;

public class SQLToXML {

  public static void main(String[] args ) {

    // Load the ODBC driver
    try {
      Class.forName( "sun.jdbc.odbc.JdbcOdbcDriver" );
    }
    catch (ClassNotFoundException e) {
      System.err.println("Could not load the JDBC-ODBC Bridge");
      return;
    }

    try {
      Writer out = new OutputStreamWriter(System.out, "UTF8");
      out.write("<?xml version=\"1.0\"?>\r\n");
      out.write("<Budget>\r\n");
      writeAgencies(out);
      out.write("</Budget>\r\n");
      out.close();
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

  private static void writeAgencies(Writer out)
   throws IOException {

    Connection conn = null;
    Statement stmnt = null;
    try {
      conn = DriverManager.getConnection(
       "jdbc:odbc:budauth", "", "");
      stmnt = conn.createStatement();
      String query = "SELECT DISTINCT AgencyName, AgencyCode"
       + " FROM BudgetAuthorizationTable;";
      ResultSet agencies = stmnt.executeQuery( query );

      while( agencies.next() ) {

        String agencyName = agencies.getString("AgencyName");
        agencyName = escapeText(agencyName);
        String agencyCode = agencies.getString("AgencyCode");
        out.write("  <Agency>\r\n");
        out.write("    <Name>" + agencyName + "</Name>\r\n");
        out.write("    <Code>" + agencyCode + "</Code>\r\n");
        writeBureaus(out, conn, agencyCode);
        out.write("  </Agency>\r\n");

      }
    }
    catch (SQLException e) {
      System.err.println(e);
      e.printStackTrace();
    }
    finally {
      try {
        stmnt.close();
        conn.close();
      }
      catch(SQLException e) {
        System.err.println(e);
      }
    }

  }

  private static void writeBureaus(Writer out, Connection conn,
   String agencyCode) throws IOException, SQLException {

    String query
     = "SELECT DISTINCT BureauName, BureauCode "
     + "FROM BudgetAuthorizationTable WHERE AgencyCode='"
     + agencyCode + "';";
    Statement stmnt = conn.createStatement();
    ResultSet bureaus = stmnt.executeQuery(query);

    while( bureaus.next() ) {
      String bureauName = bureaus.getString("BureauName");
      bureauName = escapeText(bureauName);
      String bureauCode = bureaus.getString("BureauCode");
      out.write("    <Bureau>\r\n");
      out.write("      <Name>" + bureauName + "</Name>\r\n");
      out.write("      <Code>" + bureauCode + "</Code>\r\n");
      writeAccounts(out, conn, agencyCode, bureauCode);
      out.write("    </Bureau>\r\n");
      out.flush();
    }

  }

  private static void writeAccounts(Writer out, Connection conn,
   String agencyCode, String bureauCode)
   throws IOException, SQLException {

    String query = "SELECT DISTINCT AccountName, AccountCode "
     + "FROM BudgetAuthorizationTable WHERE AgencyCode='"
     + agencyCode + "' AND BureauCode='" + bureauCode + "';";
    Statement stmnt = conn.createStatement();
    ResultSet accounts = stmnt.executeQuery(query);

    while( accounts.next() ) {
      String accountName = accounts.getString("AccountName");
      accountName = escapeText(accountName);
      String accountCode = accounts.getString("AccountCode");
      out.write("      <Account>\r\n");
      out.write("        <Name>" + accountName + "</Name>\r\n");
      out.write("        <Code>" + accountCode + "</Code>\r\n");
      writeSubfunctions(
       out, conn, agencyCode, bureauCode, accountCode
      );
      out.write("      </Account>\r\n");
      out.flush();
    }

  }

  private static void writeSubfunctions(Writer out,
   Connection conn, String agencyCode, String bureauCode,
   String accountCode) throws IOException, SQLException {

    String query = "SELECT * FROM BudgetAuthorizationTable"
     + " WHERE AgencyCode='" + agencyCode + "' AND BureauCode='"
     + bureauCode + "' AND AccountCode='" + accountCode + "';";
    Statement stmnt = conn.createStatement();
    ResultSet subfunctions = stmnt.executeQuery(query);

    while( subfunctions.next() ) {
      String subfunctionTitle
       = subfunctions.getString("SubfunctionTitle");
      subfunctionTitle = escapeText(subfunctionTitle);
      String subfunctionCode
       = subfunctions.getString("SubfunctionCode");
      out.write("        <Subfunction>\r\n");
      out.write("          <Name>");
      out.write(subfunctionTitle);
      out.write("</Name>\r\n");
      out.write("          <Code>");
      out.write(subfunctionCode);
      out.write("</Code>\r\n");
      out.write("          <Amount year='TransitionQuarter'>");
      out.write(subfunctions.getInt("TransitionQuarter")
       + "</Amount>\r\n");
      for (int year = 1976; year <= 2006; year++) {
        String name = "FY" + year;
        long amt = subfunctions.getInt(name) * 1000L;
        out.write("          <Amount year='" + year + "'>");
        out.write(amt + "</Amount>\r\n");
      }
      out.write("        </Subfunction>\r\n");
      out.flush();
    }

  }

  public static String escapeText(String s) {

    if (s.indexOf('&') != -1 || s.indexOf('<') != -1
     || s.indexOf('>') != -1 || s.indexOf('"') != -1
     || s.indexOf('\'') != -1 ) {
      StringBuffer result = new StringBuffer(s.length() + 6);
      for (int i = 0; i < s.length(); i++) {
        char c = s.charAt(i);
        if (c == '&') result.append("&amp;");
        else if (c == '<') result.append("&lt;");
        else if (c == '"') result.append("&quot;");
        else if (c == '\'') result.append("&apos;");
        else if (c == '>') result.append("&gt;");
        else result.append(c);
      }
      return result.toString();
    }
    else {
      return s;
    }

  }

}

The basic approach here should be quite familiar by now. Tags are stored in string literals. These tags are written onto a Writer along with element content and attribute values that have been read from the input. The difference in this case is that since the input comes from a relational database, the program can use SQL to get the input it wants when it wants it. The Java program does not need to put itself out to accommodate the order of the input data. In essence this program is nothing more than four nested loops. The outermost loop iterates over the different agencies. This contains a loop that iterates over the bureaus within that agency. This contains a loop that iterates over the accounts within that bureau. This contains the innermost loop that iterates over the subfunctions within the account. Compare this to the contortions the other programs had to perform. Only the XQuery solution was as straightforward as this, which is not surprising since it also allows the client to specify when it wants to receive which data.

Prev | Up | Next
Alternatives to Java | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 25, 2002


Summary
Prev Chapter 4. Converting Flat Files to XML Next

Summary

Converting data in non-XML formats to XML is not particularly difficult. However, there are no magic bullets to do this for you. You just have to roll your sleeves up and write the code. Once you’ve parsed the input data and organized it in the form you want, outputting it as XML is not hard. However, parsing the non-XML input data can be quite a challenge.

Unless the data is truly flat (and real-world data very rarely is), you’ll generally want to arrange your XML markup to indicate the hierarchy and structure of the data. There are many different ways to perform the conversion that differ primarily in where the work is done. If you’re extracting data from a relational database using JDBC, you may be able to make multiple SQL queries, possibly joining tables, so that the data that enters your program is already in more or less the form and order you want. Alternately, if the input is coming from flat files, you may read it into a flat structure such as a list or an array, and then use Java code to rearrange it in a more hierarchical structure such as a tree. Finally, you can read the data in the most naive format possible, write it out again as almost an XML-copy of the original structure, then post-process the initial XML document with XSLT or XQuery to get the structure you want. All three approaches produce the same document in the end. They differ primarily in when the hard work is done: at the beginning, middle, or end. Which one to choose depends largely on your relative comfort with SQL, Java, and XSLT.

Prev | Up | Next
Relational Databases | Home | Chapter 5. Reading XML
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified August 21, 2001


Chapter 5. Reading XML
Prev Next

Chapter 5. Reading XML

Table of Contents

InputStreams and Readers
XML Parsers
Choosing an XML API
Choosing an XML Parser
Available Parsers
SAX
DOM
JAXP
JDOM
dom4j
ElectricXML
XMLPULL
Summary

Writing XML documents is very straightforward, as I hope the last two chapters proved. Reading them is not nearly as simple. Fortunately, you don’t have to do all the work yourself. You can use an XML parser to read the document for you. The XML parser exposes the contents of an XML document through an API. A client application reads an XML document through this API. As well as reading the document and providing the contents to the client application, the parser also checks the document for well-formedness and (optionally) validity. If it finds an error, it informs the client application.

InputStreams and Readers

It’s time to reverse the examples of previous two chapters. Instead of putting information into an XML document, I’m going to take it out of one. In particular, I’m going to use an example that reads the response from the Fibonacci XML-RPC servlet introduced in Chapter 3. This document has the form shown in Example 5.1.

Example 5.1. A response from the Fibonacci XML-RPC server

<?xml version="1.0"?>
<methodResponse>
  <params>
    <param>
      <value><double>28657</double></value>
    </param>
  </params>
</methodResponse>

The clients for the XML-RPC server developed in Chapter 3 simply printed the entire document on the console. Now I want to extract just the answer and strip out all the markup. That is, the user interface will look something like this:

C:\XMLJAVA>java FibonacciClient 9
34

From the user’s perspective, the XML is completely hidden. The user neither knows nor cares that the request is being sent and the response received in an XML document. That’s merely an implementation detail. In fact, the user may not even know that the request is being sent over the network rather than being processed locally. All the user sees is the very basic command line interface. Obviously you could attach a fancier GUI front-end, but since this is not a book about GUI programming, I’ll leave that as an exercise for the reader.

Given that you’re writing a client to talk to an XML-RPC server, you know that the documents you’re processing always take this form. You know that the root element is methodResponse. You know that the methodResponse element contains a single params element that in turn contains a param element. You know that this param element contains a single value element. (For the moment, I'm going to ignore the possibility of a fault response to keep the examples smaller and simpler. Adding that would be straightforward, and we'll do that in later chapters.) All of this is specified by the XML-RPC specification. If any of this is violated in the response you get back from the server, then that server is not sending correct XML-RPC. You’d probably respond to this by throwing an exception.

Given that you’re writing a client to talk to the specific servlet at http://www.elharo.com/fibonacci/XML-RPC, you know that the value element contains a single double element that in turn contains a string representing a double. This isn’t true for all XML-RPC servers, but it is for this one. If the server returns a value with a type other than double, you’d probably respond by throwing an exception, just as you would if a local method you expected to return a Double instead returned a String. The only significant difference is that in the XML-RPC case neither the compiler nor the virtual machine can do any type checking. Thus you may want to be a little bit more explicit about handling the case where something unexpected is returned.

The main point is this: most programs you write are going to read documents written in a specific XML vocabulary. They are not going to be designed to handle absolutely any well-formed document that comes down the pipe. Your programs will make assumptions about the content and structure of those documents, just as they now make assumptions about the content and structure of external objects. If you are concerned that your assumptions may occasionally be violated (and you should be), you can validate your documents against a schema of some kind so you know up-front if you’re being fed bad data. However, you do need to make some assumptions about the format of your documents before you can reasonably process them.

It’s simple enough to hook up an InputStream and/or an InputStreamReader to the document, and read it out. For example, this method reads an input XML document from the specified input stream and copies it to System.out:

public printXML(InputStream xml) {

  int c;
  while ((c = xml.read()) != -1) System.out.write(c);

}

To actually extract the information a little more work is required. You need to determine which pieces of the input you actually want and separate those out from all the rest of the text. In the Fibonacci XML-RPC example, you need to extract the text string between the <double> and </double> tags and then convert it to a java.math.BigInteger object. (Remember, I’m only using a double here because XML-RPC’s ints aren’t big enough to handle Fibonacci numbers. However, all the responses should contain an integral value.)

The readFibonacciXMLRPCResponse() method in Example 5.2 does exactly this by first reading the entire XML document into a StringBuffer, converting the buffer to a String, and then using the indexOf() and substring() methods to extract the desired information. The main() method connects to the server using the URL and URLConnection classes, sends a request document to the server using the OutputStream and OutputStreamWriter classes, and passes InputStream containing the response XML document to the readFibonacciXMLRPCResponse() method.

Example 5.2. Reading an XML-RPC Response

import java.net.*;
import java.io.*;
import java.math.BigInteger;

public class FibonacciClient {

  static String defaultServer
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java FibonacciClient number url"
      );
      return;
    }

    String server = defaultServer;
    if (args.length >= 2) server = args[1];

    try {
      // Connect to the server
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();
      Writer wout = new OutputStreamWriter(out);

      // Write the request
      wout.write("<?xml version=\"1.0\"?>\r\n");
      wout.write("<methodCall>\r\n");
      wout.write(
       "  <methodName>calculateFibonacci</methodName>\r\n");
      wout.write("  <params>\r\n");
      wout.write("    <param>\r\n");
      wout.write("      <value><int>" + args[0]
       + "</int></value>\r\n");
      wout.write("    </param>\r\n");
      wout.write("  </params>\r\n");
      wout.write("</methodCall>\r\n");

      wout.flush();
      wout.close();

      // Read the response
      InputStream in = connection.getInputStream();
      BigInteger result = readFibonacciXMLRPCResponse(in);
      System.out.println(result);

      in.close();
      connection.disconnect();
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

  private static BigInteger readFibonacciXMLRPCResponse(
   InputStream in) throws IOException, NumberFormatException,
   StringIndexOutOfBoundsException {

    StringBuffer sb = new StringBuffer();
    Reader reader = new InputStreamReader(in, "UTF-8");
    int c;
    while ((c = in.read()) != -1) sb.append((char) c);

    String document = sb.toString();
    String startTag = "<value><double>";
    String endTag = "</double></value>";
    int start = document.indexOf(startTag) + startTag.length();
    int end = document.indexOf(endTag);
    String result = document.substring(start, end);
    return new BigInteger(result);

  }

}

Reading the response XML document is more work than writing the request document, but still plausible. However, this stream- and string-based solution is far from robust. In particular, it will fail if:

  • The document returned is encoded in UTF-16 instead of UTF-8

  • An earlier part of the document contains the text “<value><double>”, even in a comment.

  • The response is written with line breaks between the value and double tags like this:

    <value>
  <double>28657</double>
</value>

  • There’s extra white space inside the double tags like this:

    <double >28657</double >

Perhaps worse than these are all the malformed responses FibonacciClient will accept even though it should recognize and reject them. And this is a simple example where we just want one piece of data that’s clearly marked up. The more data you want from an XML document, and the more complex and flexible the markup, the harder it is to find using basic string matching or even the regular expressions introduced in Java 1.4.

Straight text parsing is not the appropriate tool with which to navigate an XML document. The structure and semantics of an XML document is encoded in the document’s markup, its tags and its attributes; and you need a tool that is designed to recognize and understand this structure as well as reporting any possible errors in this structure. This tool is called an XML parser.

Prev | Up | Next
Summary | Home | XML Parsers
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 16, 2001


XML Parsers
Prev Chapter 5. Reading XML Next

XML Parsers

In order to avoid the difficulties inherent in parsing raw XML input, almost all programs that need to process XML documents rely on an XML parser to actually read the document. The parser is a software library (in Java it’s a class) that reads the XML document and checks it for well-formedness. Client applications use method calls defined in the parser API to receive or request information the parser retrieves from the XML document.

The parser shields the client application from all the complex and not particularly relevant details of XML including:

  • Transcoding the document to Unicode

  • Assembling the different parts of a document divided into multiple entities.

  • Resolving character references

  • Understanding CDATA sections

  • Checking hundreds of well-formedness constraints

  • Maintaining a list of the namespaces in-scope on each element.

  • Validating the document against its DTD or schema

  • Associating unparsed entities with particular URLs and notations

  • Assigning types to attributes

One of the original goals of XML was that it be simple enough that a “Desperate Perl Hacker” (DPH) be able to write an XML parser. The exact interpretation of this requirement varied from person to person. On one extreme, the DPH was assumed to be a Web designer accustomed to writing CGI scripts without any formal training in programming who was going to hack it together in a weekend. On the other extreme, the DPH was assumed to be Larry Wall and he was allowed two months for the task. The middle ground was a smart grad student and a couple of weeks.

Whichever way you interpreted the requirement, it wasn’t met. In fact, it took Larry Wall more than a couple of months just to add the Unicode support to Perl that XML assumed. Java developers already had adequate Unicode support, however; and thus Java parsers were a lot faster out the gate. Nonetheless, it still probably isn’t possible to write a fully conformant XML parser in a weekend, even in Java. Fortunately, however, you don’t need to. There are several dozen XML parsers available under a variety of licenses that you can use. In 2002, there’s very little need for any programmer to write their own parser. Unless you have very unusual requirements, the chance that you can write a better parser than Sun, IBM, the Apache XML Project, and numerous others have already written is quite small.

Java 1.4 is the first version of Java to include an XML parser as a standard feature. In earlier versions of Java, you need to download a parser from the Web and install it in the usual way, typically by putting its .jar file in your jre/lib/ext directory. Even in Java 1.4, you may well want to replace the standard parser with a different one that provides additional features or is simply faster on your documents.

Caution

If you’re using Windows, then chances are good you have two different ext directories, one where you installed the JDK such as C:\jdk1.3.1\jre\lib\ext and one in your Program Files folder, probably C:\Program Files\Javasoft\jre\1.3.1\lib\ext. The first is used for compiling Java programs, the second for running them. To install a new class library you need to place the relevant JAR file in both directories. It is not sufficient to place the JAR archive in one and a shortcut in the other. You need to place full copies in each ext directory.

Choosing an XML API

The most important decision you'll make at the start of an XML project is the application programming interface (API) you'll use. Many APIs are implemented by multiple vendors, so if the specific parser gives you trouble you can swap in an alternative, often without even recompiling your code. However, if you choose the wrong API, changing to a different one may well involve redesigning and rebuilding the entire application from scratch. Of course, as Fred Brooks taught us, “In most projects, the first system built is barely usable. It may be too slow, too big, awkward to use, or all three. There is no alternative but to start again, smarting but smarter, and build a redesigned version in which these problems are solved.… Hence plan to throw one away; you will, anyhow. ” [1] Still, it is much easier to change parsers than APIs.

There are two major standard APIs for processing XML documents with Java, the Simple API for XML (SAX) and the Document Object Model (DOM), each of which comes in several versions. In addition there are a host of other, somewhat idiosyncratic APIs including JDOM, dom4j, ElectricXML, and XMLPULL. Finally each specific parser generally has a native API that it exposes below the level of the standard APIs. For instance, the Xerces parser has the Xerces Native Interface (XNI). However, picking such an API limits your choice of parser, and indeed may even tie you to one particular version of the parser since parser vendors tend not to worry a great deal about maintaining native compatibility between releases. Each of these APIs has its own strengths and weaknesses.

SAX

SAX, the Simple API for XML, is the gold standard of XML APIs. It is the most complete and correct by far. Given a fully validating parser that supports all its optional features, there is very little you can’t do with it. It has one or two holes, but they're really off in the weeds of the XML specifications, and you have to look pretty hard to find them. SAX is a event driven API. The SAX classes and interfaces model the parser, the stream from which the document is read, and the client application receiving data from the parser. However, no class models the XML document itself. Instead the parser feeds content to the client application through a callback interface, much like the ones used in Swing and the AWT. This makes SAX very fast and very memory efficient (since it doesn’t have to store the entire document in memory). However, SAX programs can be harder to design and code because you normally need to develop your own data structures to hold the content from the document.

SAX works best when your processing is fairly local; that is, when all the information you need to use is close together in the document. For example, you might process one element at a time. Applications that require access to the entire document at once in order to take useful action would be better served by one of the tree-based APIs like DOM or JDOM. Finally, because SAX is so efficient, it’s the only real choice for truly huge XML documents. Of course, “truly huge” has to be defined relative to available memory. However, if the documents you're processing are in the gigabyte range, you really have no choice but to use SAX.

DOM

DOM, the Document Object Model, is a fairly complex API that models an XML document as a tree. Unlike SAX, DOM is a read-write API. It can both parse existing XML documents and create new ones. Each XML document is represented as Document object. Documents are searched, queried, and updated by invoking methods on this Document object and the objects it contains. This makes DOM much more convenient when random access to widely separated parts of the original document is required. However, it is quite memory intensive compared to SAX, and not nearly as well suited to streaming applications.

JAXP

JAXP, the Java API for XML Processing, bundles SAX and DOM together along with some factory classes and the TrAX XSLT API. (TrAX is not a general purpose XML API like SAX and DOM. I'll get to it in Chapter 17.) It is a standard part of Java 1.4 and later. However, it is not really a different API. When starting a new program, you ask yourself whether you should choose SAX or DOM. You don’t ask yourself whether you should use SAX or JAXP, or DOM or JAXP. SAX and DOM are part of JAXP.

JDOM

JDOM is a Java-native tree-based API that attempts to remove a lot of DOM’s ugliness. The JDOM mission statement is, “There is no compelling reason for a Java API to manipulate XML to be complex, tricky, unintuitive, or a pain in the neck,” and for the most part JDOM delivers. Like DOM, JDOM reads the entire document into memory before it begins to work on it; and the broad outline of JDOM programs tends to be the same as for DOM programs. However, the low-level code is a lot less tricky and ugly than the DOM equivalent. JDOM uses concrete classes and constructors rather than interfaces and factory methods. It uses standard Java coding conventions, methods, and classes throughout. JDOM programs often flow a lot more naturally than the equivalent DOM program.

I think JDOM often does make the easy problems easier; but in my experience JDOM also makes the hard problems harder. Its design shows a very solid understanding of Java, but the XML side of the equation feels much rougher. It’s missing some crucial pieces like a common node interface or superclass for navigation. JDOM works well (and much better than DOM) on fairly simple documents with no recursion, limited mixed content, and a well-known vocabulary. It begins to show some weakness when asked to process arbitrary XML. When I need to write programs that operate on any XML document, I tend to find DOM simpler despite its ugliness.

dom4j

dom4j was forked from the JDOM project fairly early on. Like JDOM, it is a Java-native, tree-based, read-write API for processing generic XML. However, it uses interfaces and factory methods rather than concrete classes and constructors. This gives you the ability to plug in your own node classes that put XML veneers on other forms of data such as objects or database records. (In theory, you could do this with DOM interfaces too; but in practice most DOM implementations are too tightly coupled to interoperate with each other’s classes.) It does have a generic node type that can be used for navigation.

ElectricXML

ElectricXML is yet another tree-based API for processing XML documents with Java. It’s quite small, which makes it suitable for use in applets and other storage limited environments. It’s the only API I mention here that isn’t open source, and the only one that requires its own parser rather than being able to plug into multiple different parsers. It’s gained a reputation as a particularly easy-to-use API. However, I’m afraid its perceived ease-of-use often stems from catering to developers’ misconceptions about XML. It is far and away the least correct of the tree-based APIs. For instance, it tends to throw away a lot of white space it shouldn’t; and its namespace handling is poorly designed. Ideally, an XML API should be as simple as it can be and no simpler. In particular, it should not be simpler than XML itself is. ElectricXML pretends that XML is less complex than it really is, which may work for a while as long as your needs are simple, but will ultimately fail when you encounter more complex documents. The only reason I mention it here is because the flaws in its design aren’t always apparent to casual users; and I tend to get a lot of e-mail from ElectricXML users asking me why I’m ignoring it.

XMLPULL

SAX is fast and very efficient, but its callback nature is uncomfortable for some programmers. Recently some effort has gone into developing pull parsers that can read streaming content like SAX does, but only when the client application requests it. The recently published standard API for such parsers is XMLPULL. XMLPULL shows promise for the future (especially for developers who need to read large documents quickly but just don’t like callbacks). However, pull parsing is still clearly in its infancy. On the XML side, namespace support is turned off by default. Even worse, XMLPULL ignores the DOCTYPE declaration, even the internal DTD subset, unless you specifically ask it to read it. From the Java side of things, XMLPULL does not take advantage of polymorphism, relying instead on such un-OOP constructs as int type codes to distinguish nodes instead of making them instances of different classes or interfaces. I don’t think XMLPULL is ready for prime time quite yet. However, none of this is unusual for such a new technology. Some of the flaws I cite were also present in earlier versions of SAX, DOM, and JDOM and were only corrected in later releases. In the next couple of years, as pull parsing evolves, XMLPULL may become a much more serious competitor to SAX.

Data Binding

Recently, there’s been a flood of so-called data binding APIs that try to map XML documents into Java classes. While DOM, JDOM, and dom4j all map XML documents into Java classes, these data binding APIs attempt to go further, mapping a Book document into a Book object rather than just a generic Document object, for example. These are sometimes useful in very limited and predictable domains. However, they tend to make too many assumptions that simply aren’t true in the general case to make them broadly suitable for XML processing. In particular, these products tend to implicitly depend on one or more of the following common fallacies:

  • Documents have schemas or DTDs.

  • Documents that do have schemas and/or DTDs are valid.

  • Structures are fairly flat and definitely not recursive; that is, they look pretty much like tables.

  • Narrative documents aren’t worth considering.

  • Mixed content doesn’t exist.

  • Choices don’t exist; that is, elements with the same name tend to have the same children.

  • Order doesn’t matter.

The fundamental flaw in these schemes is an insistence on seeing the world through object-colored glasses. XML documents can be used for object serialization, and in that use-case all these assumptions are reasonably accurate; but XML is a lot more general than that. The large majority of XML documents cannot be plausibly understood as serialized objects, though a lot of programmers approach it from that point of view because that’s what they’re familiar with. When you're an expert with a hammer, it’s not surprising that world looks like it’s full of nails.

The fact is, XML documents are not objects and schemas are not classes. The constraints and structures that apply to objects simply do not apply to XML elements and vice versa. Unlike Java objects, XML elements routinely violate their declared types, if indeed they even have a type in the first place. Even valid XML elements often have different content in different locations. Mixed content is quite common. Recursive content isn’t quite as common, but it does exist. A little more subtly, though even more importantly, XML structures are based on hierarchy and position rather than the explicit pointers of object systems. It is possible to map one to the other, but the resulting structures are ugly and fragile; and you tend to find that when you’re finished what you’ve accomplished is merely reinventing DOM. XML needs to be approached and understood on its own terms, not Java’s. Data binding APIs are just a little too limited to interest me, and I do not plan to treat them in this book.

Choosing an XML Parser

When choosing a parser library many factors come into play. These include what features the parser has, how much it costs, which APIs it implements, how buggy it is, and last and certainly least how fast the parser parses.

Features

The XML 1.0 specification does allow parsers some leeway in how much of the specification they implement. Parsers can be roughly divided into three categories:

  • Fully validating parsers

  • Parsers that do not validate, but do read the external DTD subset and resolve external parameter entity references in order to supply entity replacement text and assign attribute types

  • Parsers that read only the internal DTD subset and do not validate.

In practice there’s also a fourth category of parsers that reads the instance document but do not perform all the mandated well-formedness checks. Technically such parsers are not allowed by the XML specification, but there are still a lot of them out there.

If the documents you’re processing have DTDs, then you need to use a fully validating parser. You don’t necessarily have to turn on validation if you don’t want to. However, XML is designed in such a way that you really can’t be sure to get the full content of an XML document without reading its DTD. In some cases, the differences between a document whose DTD has been processed and the same document whose DTD has not been processed can be huge. For instance, a parser that reads the DTD will report default attribute values; but one that doesn’t won’t. The handling of ignorable white space can vary between a validating parser and one that merely reads the DTD but does not validate. External entity references will be expanded by a validating parser, but not necessarily by a non-validating parser. You should only use a non-validating parser if you’re confident none of the documents you’ll process carry document type declarations. One situation in which this is reasonable is a SOAP server or client, since SOAP specifically prohibits documents from using DOCTYPE declarations. (Even in that case, though, I still recommend that you check to see whether there is a DOCTYPE declaration and throw an exception if you spot one.)

Beyond the lines set out by XML 1.0, parsers also differ in their support for subsequent specifications and technologies. In 2002, all parsers worth considering support namespaces and automatically check for namespace well-formedness as well as XML 1.0 well-formedness. Most of these do allow you to disable these checks for the rare legacy documents that don’t adhere to namespaces rules. Currently Xerces and Oracle are the only Java parsers that support schema validation though other parsers are likely to add this in the future.

Some parsers also provide extra information not required for normal XML parsing. For instance, at your request, Xerces can inform you of the ELEMENT, ATTLIST, and ENTITY declarations in the DTD. Crimson will not do this, so if you need to read the DTD you’d pick Xerces over Crimson.

API support

Most of the major parsers support both SAX and DOM. However, there are a few parsers that only support SAX, and at least a couple that only support their own proprietary API. If you want to use DOM or SAX, make sure you pick a parser that can handle it. Xerces and Crimson can.

SAX actually includes a number of optional features that parsers are not required to support. These include validation, reporting comments, reporting declarations in the DTD, reporting the original text of the pre-parsed document, and more. If any of these are important to you, you’ll need to make sure that your parser supports them too.

The other APIs including JDOM and dom4j generally don’t provide parsers of their own. Instead they use an existing SAX or DOM parser to read a document which they then convert into their own tree model. Thus they can work with any convenient parser. The notable exception here is ElectricXML which does include its own built-in parser. ElectricXML is optimized for speed and size and does not interoperate well with SAX and DOM.

License

One often overlooked consideration when choosing a parser is the license under which the parser is published. Most parsers are free in the free-beer sense, and many are also free in the free-speech sense. However, license restrictions can still get in your way.

Since parsers are essentially class libraries that are dynamically linked to your code (as all Java libraries are) and since parsers are mostly released under fairly lenient licenses, you don’t have to worry about viral infections of your code with the GPL. In one case I’m aware of, Ælfred, any changes you make to the parser itself would have to be donated back to the community; but this would not affect the rest of your classes. That being said, you’ll be happier and more productive if you do donate your changes back to the communities for the more liberally licensed parsers like Xerces. It’s better to have your changes rolled into the main code base than to have to keep applying them every time a new version is released.

There actually aren’t that many parsers you can buy. If your company is really insistent about not using open source software, then you can probably talk IBM into selling you an overpriced license for their XML for Java parser (which is just an IBM-branded version of the open source Xerces). However, there isn’t a shrink-wrapped parser you can buy; nor is one really needed. The free parsers are more than adequate.

Correctness

An often overlooked criterion for choosing a parser is correctness, how much of the relevant specifications are implemented how well. All of the parsers I’ve used have had non-trivial bugs in at least some versions. However although no parser is perfect, some parsers are definitely more reliable than others.

I wish I could say that there was one or more good choices here, but the fact is that every single parser I’ve ever tried has sooner or later exhibited significant conformance bugs. Most of the time these fall into two categories:

  • Reporting correct constructs as errors

  • Failing to report incorrect syntax.

It’s hard to say which is worse. On the one hand, unnecessarily rejecting well-formed documents prevents you from handling data others send you. On the other hand, when a parser fails to report incorrect XML documents, it’s virtually guaranteed to cause problems for people and systems who receive the malformed documents and correctly reject them.

One thing I will say is that well-formedness is the most important criterion of all. To be seriously considered a parser has to be absolutely perfect in this area, and many aren’t. A parser must allow you to confidently determine whether a document is or is not well-formed. Validity errors are not quite as important, though they’re still significant. Many programs can ignore validity and consequently ignore any bugs in the validator.

Continuing downward in the hierarchy of seriousness are failures to properly implement the standard SAX and DOM APIs. A parser might correctly detect and report all well-formedness and validity errors, but fail to pass on the contents of the document. For example, it might throw away ignorable white space rather than making it available to the application. Even less serious but still important are violations of the contracts of the various public APIs. For example, DOM guarantees that each Text object read from a parsed document will contain the longest possible string of characters uninterrupted by markup. However, I have seen parsers that occasionally passed in adjacent text nodes as separate objects rather than merging them.

Java parsers are also subject to a number of edge conditions. For example, in SAX each attribute value is passed to the client application as a single string. Because the Java String class is backed by an array of chars indexed by an int, the maximum number of characters in a String is the same as the maximum size of an int, 2,147,483,647. However, there is no maximum number of characters that may appear in an attribute value. Admittedly a three gigabyte attribute value doesn’t seem too likely (perhaps a Base-64 encoded video?) and you’d probably run out of memory long before you bumped up against the maximum size of a string; but nonetheless XML doesn’t prohibit strings of such lengths and it would be nice to think that Java could at least theoretically handle all XML documents within the limits of available memory.

Efficiency

The last consideration is efficiency, how fast the parser is and how much memory it uses. Let me stress that again: efficiency should be your last concern when choosing a parser. As long as you use standard APIs and keep parser-dependent code to a minimum, you can always change the underlying parser later if the one you picked initially proves too inefficient.

The speed of parsing tends to be dominated by I/O considerations. If the XML document is served over the network, it’s entirely possible that the speed with which data can move over the network is the bottleneck, not the XML parsing at all. In situations where the XML is being read from the disk, the time to read the data can still be significant even if it’s not quite the bottleneck it is in network applications.

Anytime you’re reading data from a disk or the network, you should buffer your streams. You can buffer at the byte level with a BufferedInputStream or at the character level with a BufferedReader. Perhaps a little counter-intuitively, you can gain extra speed by double buffering with both byte and character buffers. However, most parsers are happier if you feed them a raw InputStream and let them convert the bytes to characters (parsers are normally better at detecting the right encoding than most client code) so I prefer to use just a BufferedInputStream and not a BufferedReader unless speed is very important and I’m very sure of the encoding in advance. If you don’t buffer your I/O, then total performance is going to be limited by I/O considerations no matter how fast the parser is.

Complicated programs can also be dominated by processing that happens after the document is parsed. For example, if the XML document lists store locations, and the client application is attempting to solve the traveling salesman problem for those store locations, then parsing the XML document is the least of your worries. In such a situation, changing the parser isn’t going to help very much at all. The time taken to parse a document normally grows only linearly with the size of the document.

One area where parser choice does make a significant difference is in the amount of memory used. SAX is generally quite efficient no matter which parser you pick. However, DOM is exactly the opposite. Building a DOM tree can easily eat up as much as ten times the size of the document itself. For example, given a one megabyte document, the DOM object representing it could be ten megabytes. If you’re using DOM or any other tree-based API to process large documents, you want a parser that uses as little memory as possible. The initial batch of DOM-capable parsers were not really optimized for space, but the more recent versions are doing a lot better. With some testing you should be able to find parsers that only use two to three times as much memory as the original document. Still, it’s pretty much guaranteed that the memory usage will be larger than the document itself.

Available Parsers

I now want to discuss a few of the more popular parsers and the relative advantages and disadvantages of each.

Xerces

I’ll begin with my parser of choice, Xerces-J from the Apache XML Project. This a very complete, validating parser that has the best conformance to the XML 1.0 and Namespaces in XML specifications I’ve encountered. It fully supports the SAX2 and DOM Level 2 APIs, as well as JAXP, though I have encountered a few bugs in the DOM support. The latest versions feature experimental support for parts of the DOM Level 3 working drafts. Xerces-J is highly configurable and suitable for almost any parsing needs. Xerces-J is also notable for being the only current parser to support the W3C XML Schema Language, though that support is not yet 100% complete or bug-free.

The Apache XML Project publishes Xerces-J under the very liberal open source Apache license. Essentially, you can do anything you like with it except use the Apache name in your own advertising. Xerces-J 1.x was based on IBM’s XML for Java parser, whose code base IBM donated to the Apache XML Project. Today, the relationship’s reversed and XML for Java is based on Xerces-J 2.x. However, in both versions there’s no significant technical difference between Xerces-J and XML for Java. The real differentiation is that if you work for a large company with a policy against using software from somebody you can’t easily sue, then you can probably pay IBM a few thousand dollars for a support contract for XML for Java. Otherwise, you might as well just use Xerces-J.

Note

The Apache XML Project also publishes Xerces-C, an open source XML parser written in C++, which is based on IBM’s XML for C++ product. However, since this is a book about Java, henceforth when you see the undifferentiated name “Xerces” it should be understood that I’m talking strictly about the Java version.

Crimson

Crimson, previously known as Java Project X, is the parser Sun bundles with the JDK 1.4. Crimson supports more or less the same APIs and specifications Xerces does—SAX2, DOM2, JAXP, XML 1.0, Namespaces in XML, etc. —with the notable exception of schemas. In my experience Crimson is somewhat buggier than Xerces. I’ve encountered well-formed documents that Crimson incorrectly reported as malformed, but that Xerces could parse without any problems. The bugs I’ve encountered in Xerces all related to validation, not to the more basic criterion of well-formedness.

The reason Crimson exists is because some Sun engineers disagreed with some IBM engineers about the proper internal design for an XML parser. (Also, the IBM code was so convoluted that nobody outside of IBM could figure it out.) Crimson was supposed to be significantly faster, more scalable, and more memory efficient than Xerces-J and not get soggy in milk either. However whether it’s actually faster than Xerces (much less significantly faster) is questionable. When first released, Sun claimed that Crimson was several times faster than Xerces. IBM ran the same benchmarks and got almost exactly opposite results. They claimed that Xerces was several times faster than Crimson. After a couple of weeks of hooting and hollering on several mailing lists, the true cause was tracked down. Sun had heavily optimized Crimson for the Sun virtual machine and just-in-time compiler; and they were naturally running their tests on Sun virtual machines. IBM publishes their own Java virtual machine, and they were optimizing for and benchmarking on their virtual machine. To no one’s great surprise, Sun’s optimizations didn’t perform nearly as well when run on non-Sun virtual machines; and IBM’s optimizations didn’t perform nearly as well when run on non-IBM virtual machines. As Donald Knuth wrote back in 1974 (when machines were a lot slower than they are today), “premature optimization is the root of all evil.”[2] Eventually both Sun and IBM began testing on multiple virtual machines and watching out for optimizations that were too tied to the architecture of any one virtual machine; and now both Xerces-J and Crimson seem to run about equally fast on equivalent hardware, regardless of VM.

The real benefit to Crimson is that it’s bundled with the JDK 1.4. (Crimson does work with earlier virtual machines back to Java 1.1. It just isn’t bundled with them in the default distribution.) Thus if you know you’re running in a 1.4 or later environment, you don’t have to worry about installing extra JAR archives and class libraries just to parse XML. You can write your code to the standard SAX and DOM classes and expect it to work out of the box. If you want to use a non-Crimson parser, then you can still install the JAR files for Xerces-J or some other parser and load its implementation explicitly. However, in Java 1.3 and earlier (which is the vast majority of the installed base at the time of this writing) you have to include some parser library with your own application.

Going forward, Sun and IBM are cooperating on Xerces-2 which will probably become the default parser in the JDK in a future release. Crimson is unlikely to be developed further or gain support for new technologies like XInclude and schemas.

Ælfred

The GNU Classpath Extensions Project’s Ælfred is actually two parsers, gnu.xml.aelfred2.SAXDriver and gnu.xml.aelfred2.XmlReader. SAXDriver aims for a small footprint rather than a large feature set. It supports XML 1.0 and Namespaces in XML. However, it implements the minimum part of XML 1.0 needed for conformance. For instance, it does not resolve external entities and does not validate. Furthermore, it does not make all the well-formedness checks it should make and can miss malformed documents. It supports SAX but not DOM. Its small size makes it particularly well-suited for applets. For less resource constrained environments, Ælfred provides XmlReader, a fully validating, fully conformant parser that supports both SAX and DOM.

Ælfred was originally written by the now defunct Microstar, who placed it in the public domain. David Brownell picked up development of the parser and brought it under the aegis of the GNU Classpath Extensions Project, an attempt to reimplement Sun’s Java extension libraries (the javax packages) as free software. Ælfred is published under the GNU General Public License with library exception. In brief, this means that as long as you only call Ælfred through its public API and don’t modify the source code yourself, the GPL does not infect your code.

Piccolo

Yuval Oren’s Piccolo is the latest entry into the parser arena. It is a very small, very fast, open source, non-validating XML parser. However, it does read the external DTD subset in order to apply default attribute values and resolve external entity references. Piccolo supports the SAX API exclusively. It does not have a DOM implementation.

[1] Fred Brooks, The Mythical Man-Month, Anniversary Edition, (Addison-Wesley, 1995) p. 116

[2] Donald Knuth, “Structured Programming with go to Statements”, Computing Surveys 6 (1974): 261-301

Prev | Up | Next
Chapter 5. Reading XML | Home | SAX
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 25, 2002


SAX
Prev Chapter 5. Reading XML Next

SAX

SAX, the Simple API for XML, was the first standard API shared across different XML parsers. SAX is unique among XML APIs in that it models the parser rather than the document. In particular the parser is represented as an instance of the XMLReader interface. The specific class that implements this interface varies from parser to parser. Most of the time you only access it through the common methods of the XMLReader interface.

A parser reads a document from beginning to end. As it does so it encounters start-tags, end-tags, text, comments, processing instructions, and more. In SAX, the parser tells the client application what it sees as it sees it by invoking methods in a ContentHandler object. ContentHandler is an interface the client application implements to receive notification of document content. The client application will instantiate a client-specific instance of the ContentHandler interface and register it with the XMLReader that’s going to parse the document. As the reader reads the document, it calls back to the methods in the registered ContentHandler object. The general pattern is very similar to how events are handled in the AWT and Swing.

Example 5.3 is a simple SAX program that communicates with the XML-RPC servlet from Chapter 3. It sends the request document using basic output stream techniques and then receives the response through SAX.

Example 5.3. A SAX based client for the Fibonacci XML-RPC server

import java.net.*;
import java.io.*;
import java.math.BigInteger;
import org.xml.sax.*;
import org.xml.sax.helpers.*;

public class FibonacciSAXClient {

  public final static String DEFAULT_SERVER
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java FibonacciSAXClient number url"
      );
      return;
    }

    String server = DEFAULT_SERVER;
    if (args.length >= 2) server = args[1];

    try {
      // Connect to the server
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();
      Writer wout = new OutputStreamWriter(out);

      // Transmit the request XML document
      wout.write("<?xml version=\"1.0\"?>\r\n");
      wout.write("<methodCall>\r\n");
      wout.write(
       "  <methodName>calculateFibonacci</methodName>\r\n");
      wout.write("  <params>\r\n");
      wout.write("    <param>\r\n");
      wout.write("      <value><int>" + args[0]
       + "</int></value>\r\n");
      wout.write("    </param>\r\n");
      wout.write("  </params>\r\n");
      wout.write("</methodCall>\r\n");

      wout.flush();
      wout.close();

      // Read the response XML document
      XMLReader parser = XMLReaderFactory.createXMLReader(
        "org.apache.xerces.parsers.SAXParser"
      );
      // There's a name conflict with java.net.ContentHandler
      // so we have to use the fully package qualified name.
      org.xml.sax.ContentHandler handler
       = new FibonacciHandler();
      parser.setContentHandler(handler);

      InputStream in = connection.getInputStream();
      InputSource source = new InputSource(in);
      parser.parse(source);
      System.out.println();

      in.close();
      connection.disconnect();
    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

Since SAX is a read-only API, I used the same code as before to write the request sent to the server. The code for reading the response, however, is quite different. Rather than reading directly from the stream, SAX bundles the InputStream in an InputSource, a generic wrapper for all the different things an XML document might be stored in— InputStream, Reader, URL, File, etc. This InputSource object is then passed to the parse() method of an XMLReader.

Several exceptions can be thrown at various points in this process. For instance, a IOException will be thrown if the socket connecting the client to the server is broken. A SAXException will be thrown if the org.apache.xerces.parsers.SAXParser class can’t be found somewhere in the class path. A SAXParseException will be thrown if the server returns malformed XML. For now, Example 5.3 lumps all these together in one generic catch block. Later chapters go into the different exceptions in more detail.

There’s no code in this class to actually find the double response and print it on the console. Yet, when run it produces the expected response:

C:\XMLJAVA>java FibonacciSAXClient 42
267914296

The real work of understanding and processing documents in this particular format is happening inside the ContentHandler object. The specific implementation of the ContentHandler interface used here is FibonacciHandler, shown in Example 5.4. In this case I chose to extend the DefaultHandler adapter class rather than implement the ContentHandler interface directly. The pattern is similar to using WindowAdapter instead of WindowListener in the AWT. It avoids having to implement a lot of do-nothing methods that don’t matter in this particular program.

Example 5.4. The ContentHandler for the SAX client for the Fibonacci XML-RPC server

import org.xml.sax.*;
import org.xml.sax.helpers.DefaultHandler;

public class FibonacciHandler extends DefaultHandler {

  private boolean inDouble = false;

  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {

    if (localName.equals("double")) inDouble = true;

  }

  public void endElement(String namespaceURI, String localName,
   String qualifiedName) throws SAXException {

    if (localName.equals("double")) inDouble = false;

  }

  public void characters(char[] ch, int start, int length)
  throws SAXException {

    if (inDouble) {
      for (int i = start; i < start+length; i++) {
        System.out.print(ch[i]);
      }
    }

  }

}

What this ContentHandler needs to do is recognize and print the contents of the single double element in the response while ignoring everything else. Thus, when the startElement() method, which is invoked by the parser every time it encounters a start-tag or an empty-element tag, sees a start-tag with the name double, it sets a private boolean field named inDouble to true. When the endElement() method sees an end-tag with the name double, it sets the same field back to false. The characters() method prints whatever it sees on System.out, but only when inDouble is true.

Unlike the earlier stream and string based solution, this program will detect any well-formedness errors in the document. It will not be tripped up by the unexpected appearance of <double> tags in comments or processing instructions or ignorable white space between tags. This program would not detect problems that occurred as a result of multiple double elements or other invalid markup. However, in later chapters I’ll show you how to use a schema to add this capability. The parser-based client is much more robust than Example 5.2 and it’s almost as simple. As the markup becomes more complex and the amount of information you need to extract from the document grows, parser-based solutions become far easier and cheaper to implement than any alternative.

The big advantage to SAX compared to other parser APIs is that it’s quite fast and extremely memory-efficient. You only need to store in memory those parts of the document you actually care about. You can ignore the rest. DOM, by contrast, must keep the entire document in memory at once. [3] Furthermore, the DOM data structures tend to be substantially less efficient than the serialized XML itself. A DOM Document object can easily take up ten times as much memory as would be required to just hold the characters of the document in an array. This severely limits the size of documents that can be processed with DOM and other tree-based APIs. SAX, by contrast, can handle documents that vastly exceed the amount of available memory. If your documents cross the gigabyte threshold, there is really no alternative to SAX.

Furthermore, SAX works very well in streaming applications. A SAX program can begin working with the start of a document before the parser has reached the middle. This is particularly important in low-bandwidth, high-latency environments like most network applications. For example, if a client sent a brokerage an XML document containing a list of stocks to buy, the brokerage could execute the first trade before the entire document had been received or parsed. Multi-threading can be especially useful here.

The downside to SAX is that most programs are more concerned with XML documents than with XML parsers. In other words, a class hierarchy that models the XML document is a lot more natural and closer to what you’re likely to need than a class hierarchy that models parsers. SAX programs tend to be more than a little obtuse. It’s rare that SAX gives you all the information you need at the same time. Most of the time you find yourself building data structures in memory to store the parts of the document you’re interested in until you’re ready to use them. In the worst case, you can end up inventing your own tree model for the entire document, in which case you’re probably better off just using DOM or one of the other tree models in the first place and saving yourself the work.

[3] Xerces does give you the option of using a lazy DOM that only parses parts of the document as they’re needed to help reduce memory usage.

Prev | Up | Next
XML Parsers | Home | DOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 25, 2002


DOM
Prev Chapter 5. Reading XML Next

DOM

The Document Object Model, DOM, is the second major standard API for XML parsers, and the first tree-based API I’ll consider. Most major parsers implement both SAX and DOM. DOM programs start off similarly to SAX programs, by having a parser object read an XML document from an input stream or other source. However, where the SAX parser returns the document broken up into a series of small pieces, the equivalent DOM method returns an entire Document object that contains everything in the original XML document. You read information from the document by invoking methods on this Document object or on the other objects it contains. This makes DOM much more convenient when random access to widely separated parts of the original document is required. However, it is quite memory intensive compared to SAX, and not nearly as well suited to streaming applications.

A second advantage to DOM is that it is a read-write API. Whereas SAX can only parse existing XML documents, DOM can also create them. Documents created in this fashion are automatically well-formed. Attempting to create a malformed document throws an exception. Example 5.5 is a DOM-based program for connecting to the Fibonacci XML-RPC servlet. The request is formed as a new DOM document. The response is read as a parsed DOM document.

Example 5.5. A DOM based client for the Fibonacci XML-RPC server

import java.net.*;
import java.io.*;
import org.w3c.dom.*;
import org.apache.xerces.dom.*;
import org.apache.xerces.parsers.*;
import org.apache.xml.serialize.*;
import org.xml.sax.InputSource;

public class FibonacciDOMClient {

  public final static String DEFAULT_SERVER
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java FibonacciDOMClient number url"
      );
      return;
    }

    String server = DEFAULT_SERVER;
    if (args.length >= 2) server = args[1];

    try {

      // Build the request document
      DOMImplementation impl
       = DOMImplementationImpl.getDOMImplementation();

      Document request
       = impl.createDocument(null, "methodCall", null);

      Element methodCall = request.getDocumentElement();

      Element methodName = request.createElement("methodName");
      Text text = request.createTextNode("calculateFibonacci");
      methodName.appendChild(text);
      methodCall.appendChild(methodName);

      Element params = request.createElement("params");
      methodCall.appendChild(params);

      Element param = request.createElement("param");
      params.appendChild(param);

      Element value = request.createElement("value");
      param.appendChild(value);

      // Had to break the naming convention here because of a
      // conflict with the Java keyword int
      Element intElement = request.createElement("int");
      Text index = request.createTextNode(args[0]);
      intElement.appendChild(index);
      value.appendChild(intElement);

      // Transmit the request document
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();

      OutputFormat fmt = new OutputFormat(request);
      XMLSerializer serializer = new XMLSerializer(out, fmt);
      serializer.serialize(request);

      out.flush();
      out.close();

      // Read the response
      DOMParser parser = new DOMParser();
      InputStream in = connection.getInputStream();
      InputSource source = new InputSource(in);
      parser.parse(source);
      in.close();
      connection.disconnect();

      Document doc = parser.getDocument();
      NodeList doubles = doc.getElementsByTagName("double");
      Node datum = doubles.item(0);
      Text result = (Text) datum.getFirstChild();
      System.out.println(result.getNodeValue());
    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

In DOM the request document is built as a tree. Everything in the document is a node in this tree including not only elements but also text nodes, comments, processing instructions and more. The document serves as a factory for creating the various kinds of node objects. Each node in this tree belongs to exactly one document. After being created the node object is appended to the child list of its parent node.

Once the Document object has been created and populated, it needs to be serialized onto the URLConnection’s output stream. Unfortunately, there is no standard parser independent way to do this in DOM Level 2. This will be added in DOM Level 3. In the meantime, you need to resort to parser-specific classes and methods. Here I’ve used Xerces’s org.apache.xml.serialize package. The basic design is that an XMLSerializer object is connected to an output stream. Options for serialization such as where to place line breaks and what character encoding to use are specified by an OutputFormat object. Here I just used the default OutputFormat. The document is written onto the stream using the XMLSerializer’s serialize() method.

Once the server receives and parses the request, it calculates and transmits its response as an XML document. This document must be parsed to extract the single string we actually want. DOM includes a number of methods and classes to extract particular parts of a document without necessarily walking down the entire tree. The one I use here is the Document class’s getElementsByTagName() method. This returns a NodeList containing one Node object for each element in the input document that has the name double. In this case there’s exactly one of those so it’s extracted from the list. We then get the first child of that node, which happens to be a Text node that contains the value we want. This value is retrieved by the getNodeValue() method.

The first problem with DOM should now be apparent. It’s more than a little complex, even for very simple problems like this one. However, DOM does have an internal logic; and once you become accustomed to it, you’ll find it’s actually not that hard to use. Still, the learning curve is quite steep; and frequent reference to the documentation is a necessity.

The second downside to DOM is that it does not expose as much of the information in an XML document as SAX does. Although the basic content of elements, text, and attributes is well supported by both DOM and SAX, there are many more esoteric aspects of XML documents that SAX provides but DOM does not. These include unparsed entities, notations, attribute types, and declarations in the DTD. Some of this will be provided in DOM Level 3.

The third downside to DOM is that it’s not as complete as SAX. Much of the code in Example 5.5 is actually part of the Xerces parser rather than standard DOM. Such parser specific code is virtually impossible to avoid when programming in DOM because DOM doesn’t give you any way to create a new XML document, create a new parser, or write a Document onto an output stream. All of these have to be provided by the parser. If I were to port Example 5.5 to Crimson or GNU JAXP, I’d have to rewrite about half of it. DOM3 is going to fill in a lot of these holes. However, since DOM3 is still just a working draft with little parser support, I chose to stick to DOM2 for the time being. A few of these holes can also be plugged by JAXP.

Prev | Up | Next
SAX | Home | JAXP
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 25, 2002


JAXP
Prev Chapter 5. Reading XML Next

JAXP

Starting in Java 1.4, Sun bundled the Crimson XML parser and the SAX2, DOM2, and TrAX APIs into the standard Java class library. (TrAX is an XSLT API that sits on top of XML APIs like SAX and DOM. We’ll get to it in the last chapter.) They also threw in a couple of factory classes, and called the whole thing the “Java API for XML Processing” (JAXP).

Crimson’s a reasonably fast parser and worth your consideration. The fact that this implementation is bundled with Sun’s VM is a major plus. It allows you to distribute Java programs that use XML without having to add several megabytes of your own parser and interface classes. However, API-wise there isn’t a whole lot new here. When starting a new program you ask yourself whether you should choose SAX or DOM. You don’t ask yourself whether you should use SAX or JAXP, or DOM or JAXP. SAX and DOM are part of JAXP. If you know SAX, DOM, and TrAX, you know 99% of JAXP.

The only public part of JAXP that isn’t part of its component APIs are the factory classes in javax.xml.parsers. You can use these to create new documents in memory and load existing documents from text files and streams. You can also use the TrAX API to do some simple serialization by copying a document from a DOM object to a stream. Putting them all together, JAXP can replace most of the parser-dependent part of DOM. Example 5.6 is a JAXP client for the XML-RPC server. All the DOM standard code is the same as before in Example 5.5. However, the parser-dependent parts from the org.apache packages have been replaced with JAXP classes instead.

Example 5.6. A JAXP based client for the Fibonacci XML-RPC server

import java.net.*;
import java.io.*;
import org.w3c.dom.*;
import javax.xml.parsers.*;
import javax.xml.transform.*;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.dom.DOMSource;

public class FibonacciJAXPClient {

  private static String DEFAULT_SERVER
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java FibonacciJAXPClient number url"
      );
      return;
    }

    String server = DEFAULT_SERVER;
    if (args.length >= 2) server = args[1];

    try {
      // Build the request document
      DocumentBuilderFactory builderFactory
       = DocumentBuilderFactory.newInstance();
      DocumentBuilder builder
       = builderFactory.newDocumentBuilder();
      Document request = builder.newDocument();

      Element methodCall = request.createElement("methodCall");
      request.appendChild(methodCall);

      Element methodName = request.createElement("methodName");
      Text text = request.createTextNode("calculateFibonacci");
      methodName.appendChild(text);
      methodCall.appendChild(methodName);

      Element params = request.createElement("params");
      methodCall.appendChild(params);

      Element param = request.createElement("param");
      params.appendChild(param);

      Element value = request.createElement("value");
      param.appendChild(value);

      // Had to break the naming convention here because of a
      // conflict with the Java keyword int
      Element intElement = request.createElement("int");
      Text index = request.createTextNode(args[0]);
      intElement.appendChild(index);
      value.appendChild(intElement);

      // Transmit the request document
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();

      TransformerFactory xformFactory
       = TransformerFactory.newInstance();
      Transformer idTransform = xformFactory.newTransformer();
      Source input = new DOMSource(request);
      Result output = new StreamResult(out);
      idTransform.transform(input, output);

      out.flush();
      out.close();

      // Read the response
      InputStream in = connection.getInputStream();
      Document response = builder.parse(in);
      in.close();
      connection.disconnect();

      NodeList doubles = response.getElementsByTagName("double");
      Node datum = doubles.item(0);
      Text result = (Text) datum.getFirstChild();
      System.out.println(result.getNodeValue());
    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

Once again, the request document is built as a tree. However, this time a DocumentBuilderFactory from JAXP does the building instead of the Xerces-specific DOMImplementationImpl class. Unlike DOMImplementation, DocumentBuilder creates the Document without a root element. Consequently the next step is to create the root methodCall element and append it to the Document’s list of children. The next lines after this are the same as in the FibonacciDOMClient.

When it becomes time to serialize the Document, the JAXP solution once again diverges. Here, FibonacciDOMClient used Xerces-specific classes. FibonacciJAXPClient uses TrAX. Specifically it creates a Transformer object initialized to perform an identity transformation. It sets the source for the transform to the original DOM request document and the result to the output stream connected to the server, and the document is transformed from one onto the other. It’s a little round-about, but it works.

Finally, parsing the server response is much the same as before. However, this time instead of using the Xerces-specific DOMParser class, we use the same DocumentBuilder that created the request document. DocumentBuilder may delegate the parsing to Xerces anyway, depending on which classes are where in your class path, and how certain environment variables are set. However, there’s no need for code at this level to know that implementation detail. If it becomes important to specify the parser for reasons of performance or conformance, you can set the javax.xml.parsers.DocumentBuilderFactory system property to the name of the factory class you want to load, which then chooses the parser class. If this property is not set, a reasonable default class will be used (most likely org.apache.crimson.jaxp.DocumentBuilderFactoryImpl from Crimson or org.apache.xerces.jaxp.DocumentBuilderFactoryImpl from Xerces).

The javax.xml.parsers package does fill a hole in DOM Level 2. However, it’s not as well designed as the similar classes that are coming down the road in DOM Level 3. The SAXBuilderFactory class is completely redundant with the much better designed XMLReaderFactory class that’s a standard part of SAX2. Frankly, javax.xml.parsers is just a legacy package dating from the days of SAX1 and DOM2, which did not provide this functionality. Forward-looking programmers can ignore it. What JAXP really is, is a bundle of the standard APIs. However, it is not a new API itself; and it is not an alternative to SAX or DOM.

Prev | Up | Next
DOM | Home | JDOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 25, 2002


JDOM
Prev Chapter 5. Reading XML Next

JDOM

A large portion of DOM’s complexity and unintuitiveness is a result of it not actually being a Java API. DOM is defined in the Interface Definition Language (IDL) from the Object Management Group (OMG) and then compiled to Java. This means that DOM must use interfaces and factory methods instead of classes and constructors. DOM classes must always cater to the lowest common denominator of programming languages. For example, there are no overloaded methods in DOM because JavaScript doesn’t support method overloading. That a lot of the DOM could be more concisely expressed in Java and C++ with overloaded methods doesn’t matter.

JDOM is a more intuitive, tree-based API designed just for Java. It makes no compromises to support other languages. It includes Java specific features like equals() and hashCode() methods. Where appropriate it makes free use of standard Java classes like List and String. Consequently JDOM feels a lot more intuitive to most Java programmers than DOM does. Take a look at Example 5.7. I think you’ll agree that it’s a lot easier to comprehend at first glance than the DOM equivalent.

Example 5.7. A JDOM based client for the Fibonacci XML-RPC server

import java.net.*;
import java.io.*;
import org.jdom.*;
import org.jdom.output.XMLOutputter;
import org.jdom.input.SAXBuilder;

public class FibonacciJDOMClient {

  public final static String DEFAULT_SERVER
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java FibonacciJDOMClient number url"
      );
      return;
    }

    String server = DEFAULT_SERVER;
    if (args.length >= 2) server = args[1];

    try {
      // Build the request document
      Element methodCall = new Element("methodCall");
      Element methodName = new Element("methodName");
      methodName.setText("calculateFibonacci");
      methodCall.addContent(methodName);
      Element params = new Element("params");
      methodCall.addContent(params);
      Element param = new Element("param");
      params.addContent(param);
      Element value = new Element("value");
      param.addContent(value);
      // Had to break the naming convention here because of a
      // conflict with the Java keyword int
      Element intElement = new Element("int");
      intElement.setText(args[0]);
      value.addContent(intElement);
      Document request = new Document(methodCall);

      // Transmit the request document
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();

      XMLOutputter serializer = new XMLOutputter();
      serializer.output(request, out);

      out.flush();
      out.close();

      // Read the response
      InputStream in = connection.getInputStream();
      SAXBuilder parser = new SAXBuilder();
      Document response = parser.build(in);
      in.close();
      connection.disconnect();

      // Walk down the tree
      String result = response.getRootElement()
                       .getChild("params")
                       .getChild("param")
                       .getChild("value")
                       .getChild("double")
                       .getText();
      System.out.println(result);
    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

In JDOM the request document is built as a tree. JDOM, however, uses classes and constructors rather than factory methods. Elements can be created that are not attached to any document. Elements and most other nodes are added to a document by passing them to their parent’s addContent() method. However, if an element only contains text, no markup, then its entire content may be set by passing a string to its setText() method instead.

To serialize documents JDOM provides the XMLOutputter class. This class combines both formatting options such as encoding and the actual writing of the document onto a stream or writer. This is a standard part of JDOM that is used the same way no matter which parser you use underneath JDOM.

Once the response has been received, JDOM parses it into a Document object using the SAXBuilder class. This is the class that connects JDOM to an underlying parser that implements the SAX API. JDOM does not provide convenient methods for extracting subsets of the nodes in a document as DOM does. Consequently to get to the element we want, we must walk the tree. Since the document is very structured, this is straightforward. First get the root element of the document, then get its params child, then its param child, then its value child, then its double child, and finally its text content. The result is the string value we need. Although this string value happens to represent a double, JDOM has no way of knowing this (nor does any other XML API).

JDOM is definitely easier than DOM for simple problems like this. However, I think it fails when it comes to the more complex problems such as processing large narrative documents with lots of mixed content or working with XPath based technologies like XSLT, XPointers, XInclude, and XQuery. The design of the JDOM API has often chosen simplicity and intuitiveness over power and completeness. The incomplete tree model really begins to hurt as the problems grow more complex. Any operation that requires you to recursively walk the entire document tree is more complex in JDOM than in DOM. JDOM is much better suited to data oriented documents with well-known record-like structures that do not contain any mixed content.

Prev | Up | Next
JAXP | Home | dom4j
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 09, 2002


dom4j
Prev Chapter 5. Reading XML Next

dom4j

dom4j began as a fork of the JDOM project by a developer, James Strachan, who disagreed with several aspects of the JDOM architecture including the decision to use classes instead of interfaces. dom4j has since gone its own way, and features a very different approach to a pure-Java, tree-based API than either DOM or JDOM. Notable unique features in dom4j include integrated XPath and XSLT support and optional DOM compatibility. (e.g., you can wrap a dom4j Element inside a DOM Element.)

Example 5.8 shows the dom4j version of our standard example. It’s structured pretty much like the previous two tree based APIs. First a Document object representing the XML-RPC request is constructed in memory; then serialized onto the URLConnection’s output stream. Finally, the response document is parsed to find the desired value.

Example 5.8. A dom4j based client for the Fibonacci XML-RPC server

import java.net.*;
import java.io.*;
import org.dom4j.*;
import org.dom4j.io.*;

public class Fibonaccidom4jClient {

  public final static String DEFAULT_SERVER
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java Fibonaccidom4jClient number url"
      );
      return;
    }

    String server = DEFAULT_SERVER;
    if (args.length >= 2) server = args[1];

    try {
      // Build request document
      Document request = DocumentHelper.createDocument();
      Element methodCall = request.addElement("methodCall");
      Element methodName = methodCall.addElement("methodName");
      methodName.addText("calculateFibonacci");
      Element params = methodCall.addElement("params");
      Element param = params.addElement("param");
      Element value = param.addElement("value");
      // Had to break the naming convention here because of a
      // conflict with the Java keyword int
      Element intElement = value.addElement("int");
      intElement.addText(args[0]);

      // Transmit the request document
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();

      XMLWriter serializer = new XMLWriter(out);
      serializer.write(request);

      out.flush();
      out.close();

      // Read the response
      InputStream in = connection.getInputStream();
      SAXReader reader = new SAXReader();
      Document response = reader.read(in);
      in.close();
      connection.disconnect();

      // Use XPath to find the element we want
      Node node = response.selectSingleNode(
       "/methodResponse/params/param/value/double"
      );

      String result = node.getStringValue();
      System.out.println(result);

    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

In dom4j the request Document object is initially created by the DocumentHelper.createDocument() factory method. The contents of the document are formed at the same time they’re added to their parent so that no nodes are ever orphaned from the tree. The serialization to the server is performed by the XMLWriter class.

Once the response has been received, dom4j parses it into a Document object using the SAXReader class that connects dom4j to an underlying parser such as Crimson or Xerces. XPath is used to extract the double element from this document. If you’re comfortable with XPath, this is a very useful and convenient feature.

Prev | Up | Next
JDOM | Home | ElectricXML
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 25, 2002


ElectricXML
Prev Chapter 5. Reading XML Next

ElectricXML

The Mind Electric’s ElectricXML is yet another tree-based API for processing XML documents with Java. It has a reputation for being particularly easy to use. It’s also small. The JAR archive for ElectricXML 4.0 is around 10% of the size of the JAR archive for dom4j 1.3. Finally, the Mind Electric has also claimed in the past that ElectricXML is much faster and more memory efficient than DOM and JDOM. However, their benchmarks were a little too scant and a little too contrived to convince me.

Example 5.9 shows this chapter’s standard example program implemented in ElectricXML. As with the previous examples the input number is read from the command line. This is wrapped up in an XML request document built in memory using the ElectricXML API, which is then serialized onto an output stream connected to the server. Finally the server’s response document is read and parsed.

Example 5.9. An ElectricXML based client for the Fibonacci XML-RPC server

import java.net.*;
import java.io.*;
import electric.xml.*;

public class FibonacciElectricXMLClient {

  private static String defaultServer
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java FibonacciElectricXMLClient number url"
      );
      return;
    }

    String server = defaultServer;
    if (args.length >= 2) server = args[1];

    try {
      // Build request document
      Document request = new Document();
      request.setRoot("methodCall");
      Element methodCall = request.getRoot();
      Element methodName = methodCall.addElement("methodName");
      methodName.setText("calculateFibonacci");
      Element params = methodCall.addElement("params");
      Element param = params.addElement("param");
      Element value = param.addElement("value");
      // Had to break the naming convention here because of a
      // conflict with the Java keyword int
      Element intElement = value.addElement("int");
      intElement.setText(args[0]);

      // Transmit the request documentf
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();
      request.write(out);
      out.flush();
      out.close();

      // Read the response
      InputStream in = connection.getInputStream();
      Document response = new Document(in);
      in.close();
      connection.disconnect();

      // Walk down the tree
      String result = response.getRoot()
                       .getElement("params")
                       .getElement("param")
                       .getElement("value")
                       .getElement("double")
                       .getTextString();
      System.out.println(result);

    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

By this point, you should be experiencing a sense of deja vu. The creation of the XML-RPC request document is very similar to how it’s done in dom4j. Navigating the response document to locate the double element is very similar to how this is done in JDOM. There are only so many plausible ways to design a tree-based API for processing XML. The most original part of the ElectricXML API is that no separate serializer class is used to write the document onto the OutputStream. Instead the Document, Element and other classes have a write() method. The OutputStream onto which the node will be serialized is passed to this method. Each node takes responsibility for its own serialization.

My major technical concern about ElectricXML is that its reputation for ease of use has been achieved primarily by catering to developers’ preconceptions and prejudices about XML. In other words, the API is designed around what developers think the XML specification says, rather than what it actually does say. For example, ElectricXML routinely deletes white space in element content (sometimes mistakenly called ignorable white space) even though the XML specification explicitly states, “An XML processor must always pass all characters in a document that are not markup through to the application.” The client application is free to ignore white space characters if it doesn’t need them, just like it’s free to ignore anything else it doesn’t need; but the parser is not free to make that decision for the client application. Worse yet, the ElectricXML namespace model focuses on namespace prefixes instead of namespace URIs. This certainly matches how most developers expect namespaces to work, but it is not in fact how they do work. I agree that the XML’s namespace syntax is needlessly complicated and confusing. Nonetheless, an XML API cannot fix the problem by pretending XML is less complicated than it really is. ElectricXML may feel easier at first than more XML-compatible APIs like SAX, DOM, and JDOM; but it’s bound to cause more pain in the long run.

I also have one major non-technical concern about ElectricXML. Whereas all the other APIs discussed here are released as various forms of open source, ElectricXML is not. The license limits what you’re allowed to do with the software including preventing you from competing with it, so necessary forks are prohibited. ElectricXML is still free-beer, and source code is provided.

Prev | Up | Next
dom4j | Home | XMLPULL
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 25, 2002


XMLPULL
Prev Chapter 5. Reading XML Next

XMLPULL

A pull parser works in streaming mode like SAX. However, the client specifically requests the next chunk of data from the parser when it’s ready for it, rather than taking it whenever the parser is ready to provide it. Furthermore, the client has a limited ability to filter out some normally insignificant pieces such as comments and processing instructions. This can be as fast and as memory efficient as SAX parsing while perhaps being more familiar to many programmers.

The developing standard API for pull parsing is known as XMLPULL. This is currently implemented by two parsers, Enhydra’s kXML2 and Aleksander Slominski’s MXP1. The client application loads a parser-specific instance of the XmlPullParser interface. It sets the InputStream or Reader from which the parser will read the document. However, the parser does not actually read the document until the client calls the next() method. Then the parser reads just enough to report the next start-tag, text string, or end-tag from the document. (Actually, the parser may well read quite a bit more and buffer the result, but that’s an implementation detail that’s invisible to the client.) When the client has processed that unit of data, it asks the parser for the next unit. This continues until next() indicates that it’s at the end of the document. Example 5.10 demonstrates with an XMLPULL client for the Fibonacci XML-RPC server.

Example 5.10. An XMLPULL based client for the Fibonacci XML-RPC server

import org.xmlpull.v1.*;
import java.io.*;
import java.net.*;

public class FibonacciPullClient {

  public final static String DEFAULT_SERVER
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java FibonacciPullClient number url"
      );
      return;
    }

    String server = DEFAULT_SERVER;
    if (args.length >= 2) server = args[1];

    try {
      // Connect to the server
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();
      Writer wout = new OutputStreamWriter(out);

      // Transmit the request XML document
      wout.write("<?xml version=\"1.0\"?>\r\n");
      wout.write("<methodCall>\r\n");
      wout.write(
       "  <methodName>calculateFibonacci</methodName>\r\n");
      wout.write("  <params>\r\n");
      wout.write("    <param>\r\n");
      wout.write("      <value><int>" + args[0]
       + "</int></value>\r\n");
      wout.write("    </param>\r\n");
      wout.write("  </params>\r\n");
      wout.write("</methodCall>\r\n");

      wout.flush();
      wout.close();

      // Load the parser
      XmlPullParserFactory factory
       = XmlPullParserFactory.newInstance();
      XmlPullParser parser = factory.newPullParser();

      // Read the response XML document
      InputStream in = connection.getInputStream();
      parser.setInput(in, null);
      //                  ^^^^
      // Encoding is not known; the parser should guess it based
      // on the content of the stream.

      int event;
      while (
       (event = parser.next()) != XmlPullParser.END_DOCUMENT) {
        if( event ==  XmlPullParser.START_TAG) {
          if ( "double".equals(parser.getName()) ) {
            String value = parser.nextText();
            System.out.println(value);
          } // end if
        } // end if
      } // end while

      in.close();
      connection.disconnect();

    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

Like SAX, XMLPULL is a read-only API. Thus this program begins by sending the request to the server by writing strings onto an OutputStream, just as in the SAX client. The style for reading the response is quite new, however. The pull parser’’s next() method is repeatedly invoked. Each time it returns an int type code, XmlPullParser.START_TAG, XmlPullParser.TEXT, XmlPullParser.END_TAG, and so forth. The loop exits when the next() method returns XmlPullParser.END_DOCUMENT. Until that happens, we look at each start-tag returned to see if its name is double. If it is, then the program invokes nextText() to return the complete text content of that element as a String. This is printed on System.out.

Prev | Up | Next
ElectricXML | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 09, 2002


Summary
Prev Chapter 5. Reading XML Next

Summary

Reading an XML document is a complicated, error-prone operation, which is why it’s fortunate that you never have to write the code that does this. Instead you install an XML parser class library and only access the document through the parser’s API. Numerous XML parsers for Java are available as free-beer, open source, and payware including the Apache XML Project’s Xerces, Sun’s Crimson, Enhydra’s kXML, the GNU Classpath Project’s Ælfred, and Yuval Oren’s Piccolo. Most XML parsers implement one or more standard APIs such as SAX, DOM, and XMLPULL. As long as you only write code to the standard APIs, you can change the underlying parser with relatively little effort.

Which API you pick is largely a matter of personal taste and appropriateness for the problem at hand. SAX is an event based API that presents the contents of a document from beginning to end in the order the parser encounters them. XMLPULL is a similar read-only, streaming API. However, it’s based on the Iterator design pattern rather than the Observer design pattern. That is, in XMLPULL the client has to ask the parser for the next chunk of data whereas in SAX the parser tells the client when it has a new chunk of data. Both SAX and XMLPULL can manipulate documents larger than available memory.

SAX and XMLPULL both work well for documents where the processing is local; that is, where all the information the program needs at one time tends to be close together in the document. Programs that need random access to widely separated parts of the document are often better served by one of the tree-based APIs such as DOM or JDOM. These often feel more natural to programmers than XMLPULL and SAX, mostly because DOM models the XML document while SAX and XMLPULL model the XML parser; and most programmers are more interested in what’s in the document than how the parser behaves. The flip side is that SAX and XMLPULL are much easier APIs for parser vendors to implement so that some parsers implement SAX or XMLPULL but not DOM. JAXP bundles both SAX and DOM together so they can become a standard part of the JDK. However it really isn’t a separate API in itself.

In addition to DOM, there are several alternative tree-based APIs that layer on top of an underlying parser. These include JDOM, ElectricXML, and dom4j. These all correct various perceived flaws in the DOM API and often seem easier-to-use to a Java programmer with little background in XML. However, in my experience these APIs are much more limited than DOM and SAX and leave out some important information from an XML document needed by more complex applications. Worse yet, they occasionally increase ease-of-use by catering to common preexisting programmer misconceptions about XML syntax and grammar, and thus help propagate the mistaken beliefs. SAX and DOM are both much more in sync with the broad family of XML specifications from XML 1.0 on up. These days I try to stick to standard SAX and DOM for most of my programs.

Prev | Up | Next
XMLPULL | Home | Chapter 6. SAX
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 09, 2002


Chapter 6. SAX
Prev Next

Chapter 6. SAX

Table of Contents

What is SAX?
Parsing
Callback Interfaces
Implementing ContentHandler
Using the ContentHandler
The DefaultHandler Adapter Class
Receiving Documents
Receiving Elements
Handling Attributes
Receiving Characters
Receiving Processing Instructions
Receiving Namespace Mappings
Ignorable White Space
Receiving Skipped Entities
Receiving Locators
What the ContentHandler Doesn’t Tell You
Summary

At its core, SAX, the Simple API for XML, is based on just two interfaces, the XMLReader interface that represents the parser and the ContentHandler interface that receives data from the parser. These two interfaces alone suffice for 90% of what you need to do with SAX. This chapter shows the basic operation of XMLReader and discusses ContentHandler in detail. The next chapter explores a variety of ways to customize the parsing process through the more advanced features of the XMLReader interface.

What is SAX?

The Simple API for XML, SAX, was invented in late 1997/early 1998 when Peter Murray-Rust and several authors of XML parsers written in Java decided there wasn’t much point to maintaining multiple similar yet incompatible APIs to do exactly the same thing. Murray-Rust was the first to suggest what he called “YAXPAPI”. The reason Murray-Rust wanted Yet Another XML Parser API was that he was thoroughly sick of supporting multiple, incompatible XML parsers for his parser-client application JUMBO. Instead, he wanted a standard API everyone could agree on. Parser authors Tim Bray and David Megginson quickly signed on to the project, and work began in public on the xml-dev mailing list where many people participated. Megginson wrote the initial draft of SAX. After a short beta period, SAX 1.0 was released on May 11, 1998.

SAX was designed around abstract interfaces rather than concrete classes so it could be layered on top of parsers’ existing native APIs. SAX is not the most sophisticated XML API imaginable, but that’s part of its beauty. The ease with which SAX could be implemented by many parser vendors with very different architectures contributed to its success and rapid standardization.

SAX in other languages

SAX has been unofficially ported to several other object oriented languages including C++, Visual Basic, Python, and Perl. The general patterns and names of most functions remain the same. However, the details of implementation change quite a bit. For instance, C++ doesn’t have interfaces, but does have multiple inheritance, so ContentHandler, XMLReader and the like become classes containing nothing but pure virtual functions. The C++ string classes can’t handle Unicode so parsers must use pointers to arrays of custom types such as XMLCh instead. Unfortunately, there’s no standard C++ binding for SAX so the custom classes vary from one parser to the next, and you can’t easily port C++ SAX programs between different compilers and platforms in either binary or source form.

Although supporting the “Desperate Perl Hacker” was a goal of the original XML working group, Perl has always lagged other languages quite a bit when it comes to XML. The initial problem was the lack of support for Unicode, a sine qua non for XML. Today modern Perls have decent Unicode support. To really handle XML you need at least version 5.005_52 of Perl, preferably, 5.6.1 or later and ideally 5.8.

There are several XML parsers available for Perl, though far and away the most popular is Larry Wall and Clark Cooper’s XML::Parser. This is a wrapper around James Clark’s expat XML parser written in C. However, this parser isn’t really SAX compatible though it’s used in a lot of legacy code. New projects should use XML::SAX instead.

However, even with this module, in my opinion Perl is still not as ideal a language for processing XML as you might expect. Perl’s strength is its ability to work with the implicit structure in text documents such as tab delimited text files and comma separated values files. However, XML documents tend to have very explicit structure that is easily addressed by a language like Java. Perl’s strengths don’t come into play; but you still suffer the numerous well-known disadvantages of working with Perl, The inevitable obfuscation of Perl code seems to me too high a price to pay.

Python probably has the best support for SAX and XML of any of the non-Java languages. XML parsing including a SAX port has been a standard part of Python since version 2.0. Furthermore, Python has a standard Unicode string type. This is not quite the same as Python’s regular string type, but Python’s weak typing means this isn’t nearly as big an inconvenience as it is in C++. However, the fact remains that SAX is designed in and for Java, and Java is certainly the most convenient language with which to write SAX programs.

Although SAX is very much a de facto standard, it has not gone through any formal standardization process. Its development was open to anyone interested. All you had to do was join the xml-dev mailing list and participate in the discussions. The end result was explicitly placed in the public domain. It is free to be implemented or extended by anyone for any purpose without permission from anybody. It is not copyrighted or trademarked. As far as is known, no parts of it are patented by anyone either.

In late 1999, work began on SAX2. This was a radical reformulation of SAX that, while maintaining the same basic event-oriented architecture, replaced almost every class in SAX1. The main impetus for this radical shift was the need to make SAX namespace aware. However many other new capabilities were added in SAX2 including filters and optional support for lexical events and DTDs. SAX2 was finished in May 2000, and has proven even more successful than SAX1. Indeed SAX2 is the most complete XML API available anywhere. In 2002, all major parsers that support SAX at all support SAX2. There is no reason to learn or concern yourself with the older classes and interfaces from SAX1, and henceforth I will discuss SAX2 exclusively.

For the first few years of its life, the official SAX distribution and documentation was maintained by David Megginson. However, he recently passed the torch to David Brownell who has begun work on SAX 2.1. At the time of this writing, SAX 2.1 seems unlikely to be as radical a shift relative to SAX2 as SAX2 was relative to SAX1. Version 2.1 will add a few bits of information from the XML document that are not exposed by SAX2 such as the encoding declaration. However, no SAX2 classes, interfaces, or methods will be deprecated in SAX 2.1; and only programmers with very special needs will need to concern themselves with the new functionality in SAX 2.1.

Prev | Up | Next
Summary | Home | Parsing
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


Parsing
Prev Chapter 6. SAX Next

Parsing

Parsing is the process of reading an XML document and reporting its content to a client application while checking the document for well-formedness. SAX represents parsers as instances of the XMLReader interface. The specific class that implements this interface varies from parser to parser. For example, in Xerces it’s org.apache.xerces.parsers.SAXParser. In Crimson it’s org.apache.crimson.parser.XMLReaderImpl. Most of the time you don’t construct instances of this interface directly. Instead you use the static XMLReaderFactory.createXMLReader() factory method to create a parser-specific instance of this class. Then you pass InputSource objects containing XML documents to the parse() method of XMLReader. The parser reads the document, and throws an exception if it detects any well-formedness errors.

Example 6.1 demonstrates the complete process with a simple program whose main() method parses a document found at a URL entered on the command line. If this document is well-formed, a simple message to that effect is printed on System.out. Otherwise, if the document is not well-formed, the parser throws a SAXException. If an I/O error such as a broken network connection occurs, then the parse() method throws an IOException. In this case, you don’t know whether or not the document is well-formed.

Example 6.1. A SAX program that parses a document

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.IOException;

public class SAXChecker {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java SAXChecker URL");
      return;
    }

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader();
      parser.parse(args[0]);
      System.out.println(args[0] + " is well-formed.");
    }
    catch (SAXException e) {
      System.out.println(args[0] + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + args[0]
      );
    }

  }

}

Note

Don’t forget that you’ll probably need to install a parser such as Xerces or Ælfred somewhere in your class path before you can compile or run this program. Only Java 1.4 and later include a built-in parser.

This program’s output is straightforward. For example, here’s the output I got when I first ran it across my Cafe con Leche home page:

%java SAXChecker http://www.cafeconleche.org
http://www.cafeconleche.org is not well-formed.

After I located and fixed the bugs in that document, I got this output:

%java SAXChecker http://www.cafeconleche.org
http://www.cafeconleche.org is well-formed.

However, some readers will encounter a different result when they run this program. In particular, you may get this output:

%java SAXChecker http://www.cafeconleche.org
org.xml.sax.SAXException: System property org.xml.sax.driver not specified

What this really means is that your parser has not properly customized its version of the XMLReaderFactory class. Unfortunately, far too many parsers including Xerces and Crimson fail to do this. Consequently you need to set the org.xml.sax.driver Java system property to the fully package-qualified name of the Java class for your parser. For Xerces, it’s org.apache.xerces.parsers.SAXParser. For Crimson, it’s org.apache.crimson.parser.XMLReaderImpl. For other parsers, consult the parser documentation. You can specify a one-time value for this property using the -D flag to the Java interpeter like this:

%java -Dorg.xml.sax.driver=org.apache.xerces.parsers.SAXParser
  SAXChecker http://www.cafeconleche.org/
http://www.cafeconleche.org is well-formed.
Prev | Up | Next
Chapter 6. SAX | Home | Callback Interfaces
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified October 17, 2001


Callback Interfaces
Prev Chapter 6. SAX Next

Callback Interfaces

SAX uses the Observer design pattern to tell client applications what’s in a document.[1] Java developers are most familiar with this pattern from the event architecture of the AWT and Swing. In that context, the client programmer implements an interface such as MouseListener that receives events through well-known methods. Then the programmer registers the MouseListener object with the component such as a Button using the setMouseListener() method. When the end user moves or clicks the mouse in the button’s area, the button invokes a method in the registered MouseListener object. In this example, the Button class plays the role of the Subject, the MouseListener interface plays the role of the Observer, and the client-defined implementation of that interface plays the role of the ConcreteObserver.

SAX works in a very similar way. However in SAX, XMLReader plays the role of Subject and the org.xml.sax.ContentHandler interface plays the role of Observer. The biggest difference between the AWT and SAX is that SAX does not allow more than one listener to be registered with each XMLReader. Otherwise, the pattern is exactly the same.

Example 6.2 shows the SAX ContentHandler interface. Almost any significant SAX program you write is going to use this interface in one form or another.

Example 6.2. The SAX ContentHandler interface

package org.xml.sax;

public interface ContentHandler {

  public void setDocumentLocator(Locator locator);
  public void startDocument() throws SAXException;
  public void endDocument() throws SAXException;
  public void startPrefixMapping(String prefix, String uri)
   throws SAXException;
  public void endPrefixMapping(String prefix)
   throws SAXException;
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException;
  public void endElement(String namespaceURI, String localName,
   String qualifiedName) throws SAXException;
  public void characters(char[] text, int start, int length)
   throws SAXException;
  public void ignorableWhitespace(char[] text, int start,
   int length) throws SAXException;
  public void processingInstruction(String target, String data)
   throws SAXException;
  public void skippedEntity(String name)
   throws SAXException;

}

The ContentHandler interface declares eleven methods. As the parser—that is, the XMLReader—reads a document, it invokes the methods in this interface. When the parser reads a start-tag, it calls the startElement() method. When the parser reads some text content, it calls the characters() method. When the parser reads an end-tag, it calls the endElement() method. When the parser reads a processing instruction, it calls the processingInstruction() method. The details of what the parser’s read, e.g. the name and attributes of a start-tag, are passed as arguments to the method.

Order is maintained throughout. That is, the parser always invokes these methods in the same order it sees items in the document. In many cases, the parser calls back to these methods immediately. For example, the parser calls the startElement() method as soon as it’s read a complete start-tag. It will not read past that start-tag until the startElement() method has returned. This means you’ll generally receive some content from invalid and even malformed documents before the parser detects the error. Consequently you should be careful not to take undoable actions until you’ve reached the end of a document.

Implementing ContentHandler

A concrete example should help make this clearer. I’m going to write a very simple program that extracts all the text content from an XML document while stripping out all the tags, comments, and processing instructions. This will be divided into two parts, a class that implements ContentHandler and a class that feeds the document into the parser.

Example 6.3, TextExtractor, is the class that implements ContentHandler. It has to provide all eleven methods declared in ContentHandler. However, the only one that’s actually needed is characters(). The other ten are do-nothing methods. They have empty method bodies, and nothing happens when the parser invokes them.

Example 6.3. A SAX ContentHandler that writes all #PCDATA onto a java.io.Writer

import org.xml.sax.*;
import java.io.*;

public class TextExtractor implements ContentHandler {

  private Writer out;

  public TextExtractor(Writer out) {
    this.out = out;
  }

  public void characters(char[] text, int start, int length)
   throws SAXException {

    try {
      out.write(text, start, length);
    }
    catch (IOException e) {
      throw new SAXException(e);
    }

  }

  // do-nothing methods
  public void setDocumentLocator(Locator locator) {}
  public void startDocument() {}
  public void endDocument() {}
  public void startPrefixMapping(String prefix, String uri) {}
  public void endPrefixMapping(String prefix) {}
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) {}
  public void endElement(String namespaceURI, String localName,
   String qualifiedName) {}
  public void ignorableWhitespace(char[] text, int start,
   int length) throws SAXException {}
  public void processingInstruction(String target, String data){}
  public void skippedEntity(String name) {}

} // end TextExtractor

Besides the eleven methods declared in ContentHandler, TextExtractor has a constructor and an out field. The constructor sets this field to the Writer on which the parsed text will be output. You can always add as many additional methods, fields, and constructors as you need. You’re not limited to just those declared in the interface.

All the real work of this class happens inside characters(). When the parser reads content between tags, it passes this text to the characters() method inside an array of chars. The index of the first character of the text inside that array is given by the start argument. The number of characters is given by the length argument. In this class, the characters() method writes the sub-array of text from start to start+length onto the Writer stored in the out field.

The characters() method in this class invokes the write() method in java.io.Writer. It happens that the write() method is declared to throw an IOException. The ContentHandler interface does not declare that characters() throws IOException. Therefore this exception must be caught. However, rather than simply ignoring it or printing a pointless message on System.err, we can wrap the IOException inside SAXException, which characters() is declared to throw, and then throw that exception. This signals the parser that something went wrong, and the parser will pass the exception along to the client application. If the client application wants to know what originally went wrong, it can find out by invoking SAXException’s getException() method.

In contrast, none of the do-nothing methods such as startElement() and processingInstruction() will ever throw any exceptions. Therefore, they are not declared to throw SAXException even though ContentHandler would support this declaration. There’s no need to clutter up the code with unnecessary throws clauses, nor is it good programming practice to advertise a possible exception in the method signature when you know that exception will never occur.

Using the ContentHandler

By itself the TextExtractor class does nothing. There’s no code in the class to actually invoke any of the methods or parse a document. Although code to do this could be placed in a main() method in TextExtractor, I prefer to place it in a class of its own called ExtractorDriver which is shown in Example 6.4.

Example 6.4. The driver method for the text extractor program

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.*;

public class ExtractorDriver {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java ExtractorDriver url"
      );
      return;
    }

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader();

      // Since this just writes onto the console, it's best
      // to use the system default encoding, which is what
      // we get by not specifying an explicit encoding here.
      Writer out = new OutputStreamWriter(System.out);
      ContentHandler handler = new TextExtractor(out);
      parser.setContentHandler(handler);

      parser.parse(args[0]);

      out.flush();
    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

The main() method in this class performs the following steps:

  1. Build an instance of XMLReader using the XMLReaderFactory.createXMLReader() method.

  2. Construct a new TextExtractor object.

  3. Pass this object to the setContentHandler() method of the XMLReader.

  4. Pass the URL of the document you want to parse (read from the command line) to the XMLReader’s parse() method.

One thing to note: there’s still no code that actually invokes the characters() or any other method in the TextExtractor class! This is for the same reason that you never see any code to invoke actionPerformed() or mouseClicked() when writing GUI programs in Java. The code that actually calls these methods is hidden deep inside the class library. You rarely need to concern yourself with it directly. Here the relevant code that calls characters() is hiding somewhere inside the parser-specific implementation of the XMLReader interface.

Let’s suppose you run this program over the original XML order document, Example 1.2 from Chapter 1. The results look like this:

C:\>java ExtractorDriver order.xml

  Chez Fred

    Birdsong Clock
    244
    12
    21.95

    135 Airline Highway
    Narragansett RI 02882

  263.40
  18.44
  8.95
  290.79

The text of the original document, including white space, has been preserved. However, the markup has all been stripped. This is exactly what we asked for.

In the next few sections, we’ll explore the individual methods of the ContentHandler interface and their behavior in more detail.

The DefaultHandler Adapter Class

TextExtractor only really used one of the eleven methods declared in ContentHandler. The other ten methods were all do-nothing methods with empty bodies. In fact, few SAX programs actually use all eleven methods. Most of the time about half suffice. To take advantage of this, SAX includes the org.xml.sax.helpers.DefaultHandler convenience class that implements the ContentHandler interface (and several other callback interfaces discussed in upcoming chapters) with do-nothing methods:

public class DefaultHandler implements ContentHandler,
 DTDHandler, EntityResolver, ErrorHandler

Instead of implementing, ContentHandler directly and cluttering up your code with irrelevant methods, you can instead extend DefaultHandler. Then you only have to override those methods you actually care about, not all eleven.

For example, if TextExtractor was built on top of DefaultHandler, it would be the smaller and simpler class shown in Example 6.5.

Example 6.5. A subclass of DefaultHandler that writes all #PCDATA onto a java.io.Writer

import org.xml.sax.*;
import org.xml.sax.helpers.DefaultHandler;
import java.io.*;

public class TextExtractor extends DefaultHandler {

  private Writer out;

  public TextExtractor(Writer out) {
    this.out = out;
  }

  public void characters(char[] text, int start, int length)
   throws SAXException {
    try {
      out.write(text, start, length);
    }
    catch (IOException e) {
      throw new SAXException(e);
    }
  }

}

Programs in this book use content handlers that implement ContentHandler directly and content handlers that extend DefaultHandler, mostly depending on which subjectively feels more natural to the problem at hand. You should feel free to use whichever variation you prefer.

[1] Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides, Addison-Wesley, 1995, pp. 293-303

Prev | Up | Next
Parsing | Home | Receiving Documents
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


Receiving Documents
Prev Chapter 6. SAX Next

Receiving Documents

In general, a single XMLReader may parse multiple documents and may do so with the same ContentHandler. Consequently it’s important to tell where one document ends and the next document begins. To provide this information, the parser invokes startDocument() as soon as it begins parsing a new document before it invokes any other methods in ContentHandler. It calls endDocument() after it’s finished parsing the document, and will not report any further content from that document. No arguments are passed to either of these methods. They serve no purpose other than marking the beginning and end of a complete XML document.

Because an XMLReader may parse multiple documents with the same ContentHandler object, per-document data structures are normally initialized in the startDocument() method rather than in a constructor. These data structures can be flushed, saved, or committed as appropriate by the endDocument() method.

Caution

If you are using one ContentHandler for multiple documents, do not assume that the endDocument() method for the previous document actually ran. If one of the earlier methods such as startElement() threw an exception, it’s likely that the parsing was not finished and that any cleanup code you put in endDocument() was not executed. For safety, it’s a good idea to reinitialize all per-document data structures in startDocument().

For example, let’s revise the tag stripper program so that it can operate on multiple XML documents in series. Furthermore, rather than printing the results on a Writer we’ll store them in a List of Strings. As is common in SAX programs, we need a data structure that holds the information collected from each document. For this simple program, a simple data structure suffices, namely a StringBuffer which is stored in the currentDocument field. This field is initialized to a new StringBuffer object in the startDocument() method and converted to a string and stored in the documents vector in the endDocument() method Example 6.6 demonstrates the necessary ContentHandler class. The characters() method simply appends text to the currentDocument buffer.

Example 6.6. A ContentHandler interface that resets its data structures between documents

import org.xml.sax.*;
import java.util.List;

public class MultiTextExtractor implements ContentHandler {

  private List documents;

  // This field is deliberately not initialized in the
  // constructor. It is initialized for each document parsed, not
  // for each object constructed.
  private StringBuffer currentDocument;

  public MultiTextExtractor(List documents) {

    if (documents == null) {
      throw new NullPointerException(
       "Documents list must be non-null");
    }
    this.documents = documents;
  }

  // Initialize the per-document data structures
  public void startDocument() {

    currentDocument = new StringBuffer();

  }

  // Flush and commit the per-document data structures
  public void endDocument() {

    String text = currentDocument.toString();
    documents.add(text);

  }

  // Update the per-document data structures
  public void characters(char[] text, int start, int length) {

    currentDocument.append(text, start, length);

  }

  // do-nothing methods
  public void setDocumentLocator(Locator locator) {}
  public void startPrefixMapping(String prefix, String uri) {}
  public void endPrefixMapping(String prefix) {}
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) {}
  public void endElement(String namespaceURI, String localName,
   String qualifiedName) {}
  public void ignorableWhitespace(char[] text, int start,
   int length) {}
  public void processingInstruction(String target,
   String data) {}
  public void skippedEntity(String name) {}

}

Caution

Parsers and ContentHandlers are not thread safe or reentrant. While it’s straightforward to design a SAX program that operates on multiple documents in series, it is almost impossible to design one that operates on multiple documents in parallel. If you need to perform XML parsing in multiple, simultaneous threads, give each thread its own XMLReader and ContentHandler objects. Similarly, if you want to parse another document from inside one of the ContentHandler methods, create a new XMLReader and a new ContentHandler object to parse it with. Do not try to reuse the existing XMLReader and ContentHandler before they've finished with the current document.

Prev | Up | Next
Callback Interfaces | Home | Receiving Elements
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified February 28, 2002


Receiving Elements
Prev Chapter 6. SAX Next

Receiving Elements

In a very real sense, SAX reports tags, not elements. When the parser encounters a start-tag, it calls the startElement() method. When the parser encounters an end-tag, it calls the endElement() method. When the parser encounters an empty-element tag, it calls the startElement() method and then the endElement() method.

If an end-tag does not match its corresponding start-tag, then the parser throws a SAXParseException. Beyond that, however, you are responsible for tracking the hierarchy. For example, if you want to treat a params element inside a methodCall element differently from a params element inside a fault element, then you’ll need to store some form of state in-between calls to the startElement() and endElement() methods. This is actually quite common. Many SAX content handlers simply build up a data structure as the document is parsed, and then operate on that data structure once the document has been completely read. Provided the data structure is simpler than the XML document itself, this is a reasonable approach. However in the most general case you can find yourself inventing a complete object hierarchy to represent arbitrary XML documents. In this case, you’re better off using DOM or JDOM instead of SAX, since they’ll do the hard work of defining and building this object hierarchy for you.

The arguments to the startElement() and endElement() methods are similar:

public void startElement(String namespaceURI, String localName, String qualifiedName, Attributes atts)
    throws SAXException;
public void endElement(String namespaceURI, String localName, String qualifiedName)
    throws SAXException;

First the namespace URI is passed as a String. If the element is unqualified (i.e. it is not in a namespace), then this argument is the empty string, not null.

Next the local name is passed as a String. This is the part of the name after the prefix and the colon, if any. For instance, if an element is named SOAP-ENV:Body, then its local name is Body. However, if an element is named Body with no prefix, then its local name is still Body.

The third argument contains the qualified name as a String. This is the entire element name including the prefix and the colon, if any. For instance, if an element is named SOAP-ENV:Body, then its qualified name is SOAP-ENV:Body. However, if an element is named Body with no prefix, then its qualified name is just Body.

Finally in the startElement() method only, the set of attributes for that element is passed as a SAX-specific Attributes object. I’ll discuss this in the next section.

As an example I’m going to build a GUI representation of the tree structure of an XML document that allows you to collapse and expand the individual elements. The GUI parts will be provided by a javax.swing.JTree. The tree will be filled in startElement() and displayed in a window in endDocument(). Example 6.7 shows how.

Example 6.7. A ContentHandler class that builds a GUI representation of an XML document

import org.xml.sax.*;
import org.xml.sax.helpers.*;
import javax.swing.*;
import javax.swing.tree.*;
import java.util.*;

public class TreeViewer extends DefaultHandler {

  private Stack nodes;

  // Initialize the per-document data structures
  public void startDocument() throws SAXException {

    // The stack needs to be reinitialized for each document
    // because an exception might have interrupted parsing of a
    // previous document, leaving an unempty stack.
    nodes = new Stack();

  }

  // Make sure we always have the root element
  private TreeNode root;

  // Initialize the per-element data structures
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) {

    String data;
    if (namespaceURI.equals("")) data = localName;
    else {
      data = '{' + namespaceURI + "} " + qualifiedName;
    }
    MutableTreeNode node = new DefaultMutableTreeNode(data);
    try {
      MutableTreeNode parent = (MutableTreeNode) nodes.peek();
      parent.insert(node, parent.getChildCount());
    }
    catch (EmptyStackException e) {
      root = node;
    }
    nodes.push(node);

  }

  public void endElement(String namespaceURI, String localName,
   String qualifiedName) {
    nodes.pop();
  }

  // Flush and commit the per-document data structures
  public void endDocument() {

    JTree tree = new JTree(root);
    JScrollPane treeView = new JScrollPane(tree);
    JFrame f  = new JFrame("XML Tree");

    f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    f.getContentPane().add(treeView);
    f.pack();
    f.show();

  }

  public static void main(String[] args) {

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader(
        "org.apache.xerces.parsers.SAXParser"
      );
      ContentHandler handler = new TreeViewer();
      parser.setContentHandler(handler);
      for (int i = 0; i < args.length; i++) {
        parser.parse(args[i]);
      }
    }
    catch (Exception e) {
      System.err.println(e);
    }

  }   // end main()

} // end TreeViewer

The JTree class provides a ready-made data structure for this program. All we have to do is fill it. However, it’s also necessary to track where we are in the XML hierarchy at all times so that the parent to which the current node will be added is accessible. For this purpose a stack is very helpful. The parent element can be pushed onto the stack in startElement() and popped off the stack in endElement(). Since SAX’s beginning-to-end parsing of an XML document equates to a depth-first tree traversal, the top element in the stack always contains the most recently visited element.

I find stacks like this to be very useful in many SAX programs. More complex programs may need to build more complicated tree or object structures. If your purpose is not simply to display a GUI for the tree, then you should probably roll your own tree structure rather than using JTree as I’ve done here.

Note

TreeViewer runs with the default distribution of Java 1.2 or later. It can run with Java 1.1, but you’ll need to make sure the swingall.jar archive is somewhere in your class path. The javax.swing classes used here are not bundled with the JDK 1.1.

Figure 6.1 shows this program displaying Example 1.7 from Chapter 1. Swing allows individual parts of the tree to be collapsed or expanded; but the entire element tree is always present even if it’s hidden. JTree also allows you to customize the icons used, and even enable the user to edit the tree. However, since that’s purely Swing programming and says little to nothing about XML, I leave that as an exercise for the reader.

Figure 6.1. The Swing Based Tree Viewer

Caution

This makes a nice little example, but please don’t treat it as more than that. The tantalizing easiness of representing XML documents with widgets like java.swing.JTree and similar things in Windows, Motif, and other GUIs has spawned a lot of editors and browsers that use these tree models as user interfaces. However, not a lot of thought went into whether users actually thought of XML documents this way or could be quickly trained to do so.

In actual practice, user interfaces of this sort have failed spectacularly. A good user interface for XML editors and viewers looks a lot more like the user interfaces people are accustomed to from traditional programs such as Microsoft Word, Netscape Navigator, and Adobe Illustrator. The whole point of a GUI is that it can decouple the user interface from the underlying data model. Just because an XML document is a tree is no excuse for making users edit trees when they don’t want to.

Prev | Up | Next
Receiving Documents | Home | Handling Attributes
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 09, 2002


Handling Attributes
Prev Chapter 6. SAX Next

Handling Attributes

Attributes are not reported through separate callbacks. Instead an Attributes object containing all the attributes of an element is passed to the startElement() method for the start-tag or empty-element tag of the element that possesses the attributes. Example 6.8 summarizes the Attributes interface.

Example 6.8. The SAX Attributes interface

package org.xml.sax;

public interface Attributes {

  public int    getLength ();

  public String getQName(int index);
  public String getURI(int index);
  public String getLocalName(int index);
  public int    getIndex(String uri, String localPart);
  public int    getIndex(String qualifiedName);
  public String getType(String uri, String localName);
  public String getType(String qualifiedName);
  public String getType(int index);
  public String getValue(String uri, String localName);
  public String getValue(String qualifiedName);
  public String getValue(int index);

}

If you know the qualified name or namespace URI and local name of the attribute you want, Attributes can look up its value and type. If you don’t know the names of the attributes at compile-time, you can iterate through all the attributes of an element instead. Attributes are unordered. However, for programmer convenience the Attributes interface is designed as a list. You can ask for the value, local name, qualified name, type, and namespace URI of an attribute by giving its index into the list. Just don’t assume that the order of the attributes in this list is necessarily the same order they had in the original document. More often than not, it isn’t.

The type of the attribute is reported as one of these nine constant strings, exactly as types would be indicated in an ATTLIST declaration in a DTD:

  • CDATA

  • ID

  • IDREF

  • IDREFS

  • NMTOKEN

  • NMTOKENS

  • ENTITY

  • ENTITIES

  • NOTATION

Enumerated types are reported as having type NMTOKEN. Undeclared attributes are reported as having type CDATA. SAX does not yet support schema types such as int or gYear. Maybe in SAX 3.0.

Caution

A few parsers are not 100% compliant with the SAX specification here. In particular, Crimson and Xerces 2.0.x use the string ENUMERATION for enumerated types instead of NMTOKEN. Xerces 1.4.X reports an enumerated type as a string containing the actual enumeration, for example, ( yes | no | maybe).

If a declared attribute has any type other than CDATA, then the parser normalizes its value. This means that all tabs, carriage returns, and line feeds are converted to a single space, runs of spaces are converted to a single space, and leading and trailing white space is stripped. Only normalized values are reported by the getValue() methods. However, in order to determine an attribute type, the parser must read the DTD. If an attribute is declared in the external DTD subset, then non-validating parsers that do not read the external subset will assume the attribute has type CDATA, and fail to normalize.

If you ask an Attributes object for information (type, name, value, etc.) about an attribute that is not in that particular list, then all the methods that normally return a String return null instead. The getIndex() methods return -1. None of these methods throw any exceptions. However, if you try to use the return values without checking for null or -1 first, you’re asking for a NullPointerException or an ArrayIndexOutOfBoundsException. SAX 2.0 does not distinguish between attributes that were present in the instance document and those that were defaulted in from the DTD or schema. This may be added in SAX 2.1.

For an example, I’m going to develop a web spider that follows simple XLinks. XLink is an attribute-based syntax for embedding hypertext in arbitrary XML documents. Elements are identified as XLinks by an xlink:type attribute with the value simple. (There’s also a more powerful and more complex extended XLink which I’m going to ignore for the purposes of this example.) The URL the link points to is contained in an xlink:href attribute. The xlink prefix is mapped to the namespace URI http://www.w3.org/TR/1999/xlink. As always the prefix can change as long as the URI stays the same. For example, this is an XLink that points to The Nation’s home page:

<magazine xmlns:xlink="http://www.w3.org/TR/1999/xlink"
 xlink:type="simple" xlink:href="http://www.thenation.com/">
  The Nation
</magazine>

Note especially that the element name and content are irrelevant to the link, which is encoded purely in attributes. The link could be written like this and still indicate the same link:

<foo xmlns:xlink="http://www.w3.org/TR/1999/xlink"
 xlink:type="simple" xlink:href="http://www.thenation.com/">
  Foo
</foo>

All the information required to process the link is included in the attributes. Consequently, we can use the Attributes interface and the startElement() method to design a spider that follows XLinks. Example 6.9 is such a program. Currently this spider does nothing more than follow the links and print their URLs. However, it would not be hard to add code to load the discovered documents into a database or perform some other useful operation.

Example 6.9. A ContentHandler class that spiders XLinks

import org.xml.sax.*;
import org.xml.sax.helpers.*;
import java.io.*;
import java.util.*;

public class SAXSpider extends DefaultHandler {

  // Need to keep track of where we've been
  // so we don't get stuck in an infinite loop
  private List spideredURIs = new Vector();

  // This linked list keeps track of where we're going.
  // Although the LinkedList class does not guarantee queue like
  // access, I always access it in a first-in/first-out fashion.
  private LinkedList queue = new LinkedList();

  private String    currentURI;
  private XMLReader parser;

  public SAXSpider(XMLReader parser, String uri) {
    this.parser = parser;
    this.currentURI = uri;
  }

  public void endDocument() {

    spideredURIs.add(currentURI);
    System.out.println("Visited " + currentURI);
    String uri;
    try {
      uri = (String) queue.removeLast();
    }
    catch (NoSuchElementException e) {
      // The queue is empty; we're finished.
      return;
    }
    this.currentURI = uri;
    try {
      parser.parse(uri);
    }
    catch (Exception e) {
      // just skip this one and move on to the next
      this.endDocument();
    }

  }

  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) {

    String type
     = atts.getValue("http://www.w3.org/1999/xlink", "type");
    if (type != null) {
      String href
       = atts.getValue("http://www.w3.org/1999/xlink", "href");
      if (href != null) {
        if (!spideredURIs.contains(href)) {
          queue.addFirst(href);
        }
      }
    }

  }

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println("Usage: java SAXSpider URL1");
    }
    String uri = args[0];

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader(
        "org.apache.xerces.parsers.SAXParser"
      );

    // Install the ContentHandler
    ContentHandler spider = new SAXSpider(parser, uri);
    parser.setContentHandler(spider);
    parser.parse(uri);

    }
    catch (Exception e) {
      System.err.println(e);
    }

  } // end main

} // end SAXSpider

The startElement() method simply inspects the tag for the two relevant XLink attributes. It looks for them by namespace and local name. If it finds any and it hasn’t yet visited that URL, then it adds that URL to the end of the queue of URLs that need to be visited.

The endDocument() method prints out the URL of the document it’s just finished parsing. Then it retrieves the next URL from the top of the queue and parses it. This program is a little unusual in that not only does the XMLReader call back to the ContentHandler, but the ContentHandler calls back to its XMLReader.

The main() method reads the starting URL from the command line, constructs an XMLReader and a SAXSpider, and parses the initial URL. The program runs automatically from there. There’s no limit to the depth or number of documents this spider will search, though currently the paucity of XLinked documents on the Web makes it unlikely that this program will run forever. Furthermore, since it isn’t designed to run in parallel, there’s little chance of it overwhelming anybody’s server. Nonetheless, limiting its search depth would be a good feature to add.

Prev | Up | Next
Receiving Elements | Home | Receiving Characters
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified October 16, 2001


Receiving Characters
Prev Chapter 6. SAX Next

Receiving Characters

When the parser reads #PCDATA, it passes this text to the characters() method as an array of chars. While it would be simpler if characters() just took a String as an argument, using a char[] array allows certain performance optimizations. In particular, parsers often store a large chunk of the original document in a single array, and repeatedly pass that same array to the characters() method, while updating the values of start and length.

On the flip side, when there’s a large amount of text between two tags with no intervening markup, the parser may choose to call characters() multiple times even though it doesn’t need to. Xerces generally won’t pass more than 16K of text in one call. Crimson is limited to about 8K of text per call. At the extreme, I have even seen a parser pass a single character at a time to the characters() method. You must not assume that the parser will pass you the maximum contiguous run of text in a single call to characters().

This can lead to some uncomfortable contortions when processing many documents. Given an element such as <Name>Birdsong Clock</Name>, you typically want to process the entire content as a unit. This requires you to set a boolean flag at the start-tag for the element in startElement(), accumulate the data into a buffer of some kind, often a StringBuffer, and only act on the data when you reach the end-tag for the element as signaled by the endElement() method.

For an example, I’m going to revisit the Fibonacci XML-RPC client program from the last chapter. However, this time rather than printing the result on System.out, I’m going to collect the result and make it available as a BigInteger. Once again, this will require the ContentHandler to recognize the contents of the single double element in the response while ignoring everything else. Example 6.10 demonstrates.

Example 6.10. A SAX client for the Fibonacci XML-RPC server

import java.net.*;
import java.io.*;
import java.math.BigInteger;
import org.xml.sax.*;
import org.xml.sax.helpers.*;

public class NewFibonacciClient {

  public final static String DEFAULT_SERVER
   = "http://www.elharo.com/fibonacci/XML-RPC";

  public static BigInteger calculateFibonacci(int index,
   String server) throws IOException, SAXException {

      // Connect to the the server
      URL u = new URL(server);
      URLConnection uc = u.openConnection();
      HttpURLConnection connection = (HttpURLConnection) uc;
      connection.setDoOutput(true);
      connection.setDoInput(true);
      connection.setRequestMethod("POST");
      OutputStream out = connection.getOutputStream();
      Writer wout = new OutputStreamWriter(out, "UTF-8");

      // Transmit the request XML document
      wout.write("<?xml version=\"1.0\"?>\r\n");
      wout.write("<methodCall>\r\n");
      wout.write(
       "  <methodName>calculateFibonacci</methodName>\r\n");
      wout.write("  <params>\r\n");
      wout.write("    <param>\r\n");
      wout.write("      <value><int>" + index
       + "</int></value>\r\n");
      wout.write("    </param>\r\n");
      wout.write("  </params>\r\n");
      wout.write("</methodCall>\r\n");

      wout.flush();
      wout.close();

       // Read the response XML document
      XMLReader parser = XMLReaderFactory.createXMLReader(
        "org.apache.xerces.parsers.SAXParser"
      );
      FibonacciHandler handler = new FibonacciHandler();
      parser.setContentHandler(handler);

      InputStream in = connection.getInputStream();
      InputSource source = new InputSource(in);
      parser.parse(source);

      in.close();
      connection.disconnect();
      return handler.result;

  }

  static class FibonacciHandler extends DefaultHandler {

    StringBuffer buffer = null;
    BigInteger result = null;

    public void startElement(String namespaceURI,
     String localName, String qualifiedName, Attributes atts) {

      if (qualifiedName.equals("double")) {
        buffer = new StringBuffer();
      }

    }

    public void endElement(String namespaceURI, String localName,
     String qualifiedName) {

      if (qualifiedName.equals("double")) {
        String accumulatedText = buffer.toString();
        result = new BigInteger(accumulatedText);
        buffer = null;
      }

    }

    public void characters(char[] text, int start, int length)
     throws SAXException {

      if (buffer != null) {
        buffer.append(text, start, length);
      }

    }

  }

  public static void main(String[] args) {

    int index;
    try {
      index = Integer.parseInt(args[0]);
    }
    catch (Exception e) {
      System.out.println(
       "Usage: java NewFibonacciClient number url"
      );
      return;
    }

    String server = DEFAULT_SERVER;
    if (args.length >= 2) server = args[1];

    try {
      BigInteger result = calculateFibonacci(index, server);
      System.out.println(result);
    }
    catch (Exception e) {
      e.printStackTrace();
    }

  }

}

The return value is stored in a private BigInteger field named result. The value of this field only makes sense after the response has been received and parsed, so I hide the ContentHandler in a static inner class which is accessed through the static calculateFibonacci() method. Because ContentHandler methods often need to be called in specific order from a certain context, the strategy of hiding them inside a non-public, possibly inner class is quite common. It’s not absolutely required, but it does make the class safer and the public interface much simpler.

What’s really new here is how the characters() method operates. Fibonacci numbers grow arbitrarily large exponentially quickly. There does exist a Fibonacci number, the exact size depending on the parser, which will not be completely given in a single call to characters(). Consequently, rather than simply storing a boolean that tells us whether we’re in the double element, we use a StringBuffer field. This is null outside the double element. It is non-null inside the double element. When it is non-null, the characters() method appends data to the buffer. That data is acted on —in this case, converted to an integer in this case —only when an end-tag is spotted and the endElement() method invoked.

This general approach of accumulating data into a buffer and only acting on it after the last character of data has been seen is very common in SAX programs. Elements that contain mixed content are handled similarly. Elements that can recursively contain other elements with the same name (e.g. in XHTML a div can contain another div) are trickier, but can normally be handled by using a stack of element name flags rather than a single boolean flag. Indeed stacks are often very convenient data structures when processing XML with SAX as has been seen in earlier examples and as will be seen again before this chapter is done.

Prev | Up | Next
Handling Attributes | Home | Receiving Processing Instructions
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified February 07, 2002


Receiving Processing Instructions
Prev Chapter 6. SAX Next

Receiving Processing Instructions

Each processing instruction the parser reads is passed to the processingInstruction() method in one invocation. This includes processing instructions that occur before and after the root element. The target is given as the first argument and the data as the second argument.

public void processingInstruction(String target, String data)
    throws SAXException;

Caution

The XML declaration is technically not a processing instruction, even though it looks like one; and it is not passed to the processingInstruction() method. SAX 2.1 will add some additional features and properties for retrieving the values of the version, standalone, and encoding attributes from the XML declaration. However, until then SAX parsers do not tell client programs what was in the XML declaration or even whether the document contained an XML declaration in the first place.

Although many processing instructions such as xml-stylesheet use a pseudo-attribute format, this depends on the semantics defined for the particular target. The processingInstruction() method does not distinguish between processing instructions like xml-stylesheet that use pseudo-attributes and processing instructions like php that don’t. The data of both kinds is passed to the processingInstruction() method as a single String. If you want to split that string into attributes and values, you have to write the code to do that yourself. SAX doesn’t do it for you.

Tip

The JDOM ProcessingInstruction class is able to break up processing instructions that use pseudo-attributes into name-value pairs. Even if you’re primarily parsing with SAX, you can still use the JDOM ProcessingInstruction class to extract these pairs for processing instructions that are known to contain pseudo-attributes. For example, this SAX processingInstruction() method uses JDOM to find the value of the href attribute of an xml-stylesheet processing instruction:

  public void processingInstruction(String target, String data) {

    if (target.equals("xml-stylesheet")) {
      ProcessingInstruction xml-stylesheet
       = new ProcessingInstruction(target, data);
      String href = xml-stylesheet.getValue("href");
    }

  }

Of course for this to work, you’ll need to have the JDOM class library installed in your class path, and have imported at least org.jdom.ProcessingInstruction.

Example 6.11 demonstrates with a straightforward program to list the targets and data of all the processing instructions found in a document. Of course the real purpose of processing instructions is to pass their data to a separate program somehow identified by the target.

Example 6.11. A ContentHandler that prints processing instruction targets and data on System.out

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;

public class ProcessingInstructionLister extends DefaultHandler {

  public void processingInstruction(String target, String data) {

    System.out.println("Processing Instruction:");
    System.out.println("  target: " + target);
    System.out.println("  data:   " + data);
    System.out.println();

  }

  public static void main(String[] args) {

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader(
        "org.apache.xerces.parsers.SAXParser"
      );
      ContentHandler handler = new ProcessingInstructionLister();
      parser.setContentHandler(handler);
      for (int i = 0; i < args.length; i++) {
        parser.parse(args[i]);
      }
    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

Here’s the output from running this program over styled_order.xml from Chapter 1. This document contains a single xml-stylesheet processing instruction. Note especially that the XML declaration is not considered to be a processing instruction and is not reported.

C:\XMLJAVA>java ProcessingInstructionLister styled_order.xml
Processing Instruction:
  target: xml-stylesheet
  data:   type="text/css" href="order.css"

Prev | Up | Next
Receiving Characters | Home | Receiving Namespace Mappings
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified October 17, 2001


Receiving Namespace Mappings
Prev Chapter 6. SAX Next

Receiving Namespace Mappings

By default namespace declaration attributes such as xmlns="http://ns.cafeconleche.org/Orders/" and xmlns:xlink="http://www.w3.org/1999/xlink" are not included in the list of attributes passed to startElement(). Instead each such namespace declaration attribute is signaled by a call to startPrefixMapping() immediately before the startElement() call corresponding to the start-tag of the element where the declaration appears. Furthermore, the endElement() call corresponding to the end tag of that element is immediately followed by an endPrefixMapping() call.

public void startPrefixMapping(String prefix, String uri)
    throws SAXException;
public void endPrefixMapping(String prefix)
    throws SAXException;

90% of the time you can ignore these events. As long as you only care about URIs and prefixes on element and attribute names, then there’s really no need to pay attention to these methods. You simply inspect the URIs and prefixes passed in the arguments to the startElement() method. However, in a few applications, including XSLT, schemas, and SOAP, prefixes appear in attribute values and even element content. If so, you need to keep track of which URIs particular prefixes are bound to where. This is complicated by the fact that, while many documents declare all namespaces on the root element and most documents map each prefix to exactly one URI, it is possible for namespaces to be declared elsewhere than the root and for one prefix to match different URIs in different parts of the same document. Conversely, one URI can be associated with several prefixes, even in the same part of a document.

The proper way to manage prefix mappings is to keep the prefixes and URIs on a per-document stack. startPrefixMapping() pushes the pair onto the stack. endPrefixMapping() pops them off.[2] If at any time while parsing you need to know the current binding of a prefix you’ve encountered, simply search down the stack starting with the top. The first binding you encounter for the prefix is the one you want. The org.xml.sax.helpers.NamespaceSupport class implements this logic. It’s summarized in Example 6.12.

Example 6.12. The NamespaceSupport class

package org.xml.sax.helpers;

public class NamespaceSupport {

  public final static String XMLNS
   = "http://www.w3.org/XML/1998/namespace";

  public NamespaceSupport();

  public void        reset();
  public void        pushContext();
  public void        popContext();
  public boolean     declarePrefix(String prefix, String uri);
  public String[]    processName(String qualifedName,
   String parts[], boolean isAttribute);
  public String      getURI(String prefix);
  public Enumeration getPrefixes();
  public String      getPrefix(String uri);
  public Enumeration getPrefixes(String uri);
  public Enumeration getDeclaredPrefixes();

}

To use this class, you simply  call
pushContext() in the first
startPrefixMapping() event
before each startElement(),
declarePrefix(), and
startPrefixMapping() event,
and
popContext() in the first
endPrefixMapping() event
after each endElement() event.

You add namespace declarations to a context by passing their prefix and URI as strings to declarePrefix() inside each startPrefixMapping() call. Use the empty string as the prefix to declare the default namespace. Here’s the cookbook code you can paste into your content handler to maintain a stack of namespaces:

  private NamespaceSupport namespaces;
  private boolean needNewContext = true;

  public void startDocument() {
    namespaces = new NamespaceSupport();
  }

  public void startPrefixMapping(String prefix, String uri)
   throws SAXException {

    if (needNewContext) {
      namespaces.pushContext();
      needNewContext = false;
    }
    namespaces.declarePrefix(prefix, uri);

  }

  public void endPrefixMapping(String prefix)
   throws SAXException {

    if (needNewContext) {
      namespaces.popContext();
      needNewContext = false;
    }

   public void startElement(String namespaceURI,
    String localName, String qualifiedName, Attributes atts)
    throws SAXException {

     needNewContext = true;
     // add additional startElement() code...
  }

  }

The getDeclaredPrefixes(), getPrefix(), getPrefixes(), and getURI() methods all return various information about the namespaces in scope in the current context. For example, this startElement() method prints the namespace URI for the values of type attributes, such as might be found in a schema:

  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {

    String value = atts.getValue("type");
    if (value != null) {
      String prefix = "";
      if (value.indexOf(':') >= 0) {
        prefix = value.substring(0, value.indexOf(':'));
      }
      String uri = namespaces.getURI(prefix);
    }

  }

That point’s easy to miss so let me make it again: the getURI() method is being used here to get the namespace URI for the attribute value, not the attribute itself. The attribute in this example is in no namespace at all. NamespaceSupport and startPrefixMapping()/endPrefixMapping() are only relevant to namespace prefixes in attribute values and element content, not to namespace prefixes used on attribute and element names. If you only care about namespace prefixes in element and attribute names, then you can ignore this section completely.

If you want to reuse a NamespaceSupport for a new document, call reset(), probably from the startDocument() method.

Note

The Namespaces in XML specification explicitly states that the xml prefix is always bound to the URL http://www.w3.org/XML/1998/namespace. Even if this is explicitly declared with an xmlns:xml="http://www.w3.org/XML/1998/namespace" attribute, startPrefixMapping() and endPrefixMapping() are never invoked for the xml prefix. However, this binding is automatically included in all NamespaceSupport objects regardless of context.

Also according to Namespaces in XML, “the prefix xmlns is used only for namespace bindings and is not itself bound to any namespace name.” Thus it too does not cause startPrefixMapping() and endPrefixMapping() invocations.

[2] SAX does state that “start/endPrefixMapping events are not guaranteed to be properly nested relative to each other”. Thus you might pop mappings off in a different order than you pushed them, and thus pop the wrong mapping at the wrong time. However, as long as you push and pop all mappings as soon as you see a start/endPrefixMapping event, you’ll always pop the same set of mappings you pushed, even if in a different order. Thus this turns out not to matter in practice.

Prev | Up | Next
Receiving Processing Instructions | Home | Ignorable White Space
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified error: 0 90, 2002


Ignorable White Space
Prev Chapter 6. SAX Next

Ignorable White Space

One of the more obscure parts of the XML 1.0 specification is the perhaps misleadingly named “ignorable white space”. This is white space that occurs between tags in places where the DTD does not allow mixed content. For example, consider the XML-RPC document in Example 6.13:

Example 6.13. A document that uses ignorable white space to prettify the XML

<?xml version="1.0"?>
<!DOCTYPE methodCall [
  <!ELEMENT methodCall (methodName, params)>
  <!ELEMENT params (param+)>
  <!ELEMENT param (value)>
  <!ELEMENT value (string)>
  <!ELEMENT methodName (#PCDATA)>
  <!ELEMENT string (#PCDATA)>
]>
<methodCall>
  <methodName>lookupSymbol</methodName>
  <params>
    <param>
      <value>
        <string>
          Red Hat
        </string>
      </value>
    </param>
  </params>
</methodCall>

This example has quite a bit of white space just for indenting. In particular, the spaces, carriage returns, and line feeds between <methodCall> and <methodName>, </methodName> and <params>, <params> and <param>, <param> and <value>, </value> and </param>, </param> and </params>, and </params> and </methodCall> only exist for indenting. Furthermore, the DTD says that these elements cannot contain #PCDATA, and therefore it’s known that this white space is ignorable. Thus a validating parser will not pass these white space characters to the characters() method. Instead it passes them to the ignorableWhiteSpace() method. A non-validating parser might do the same, or it might pass the ignorable white space to the characters() method instead. If this matters to you, make sure you use a validating parser.

The space and line break characters in the string element are not ignorable because the DTD allows this element to contain #PCDATA. This white space is passed to the characters() method along with the words Red and Hat. White space is considered ignorable only where #PCDATA is invalid.

For purposes of this method, white space consists exclusively of the ASCII space (&#x20;), tab (&#x9;), carriage return (&#xD;), and line feed (&#xA;). Unicode includes many more space characters including new line (&#x85;), em space (&#x2003;), en space (&#x2002;), and more. However, these characters are never ignorable.

The ignorableWhiteSpace() method has the same arguments and the same caveats as the characters() method. For instance, there’s no guarantee that each call to this method will contain the maximum contiguous run of ignorable white space. However, its text[] argument should contain nothing except space characters, tabs, carriage returns, and linefeeds, at least in the sub-array delineated by start and start+length.

Prev | Up | Next
Receiving Namespace Mappings | Home | Receiving Skipped Entities
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified October 16, 2001


Receiving Skipped Entities
Prev Chapter 6. SAX Next

Receiving Skipped Entities

Validating parsers resolve all general entity references that occur in both element content and attribute values. However, non-validating parsers are allowed not to read the external DTD subset. For example, consider the simple XHTML document in Example 6.14.

Example 6.14. An XML document containing a potentially skipped entity reference

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
                    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <body>
     <h1>My resum&eacute;</h1>
  </body>
</html>

If a parser does not read the DTD, then it has no way of knowing what the entity reference &eacute; stands for, or indeed whether that entity reference is even properly defined. However, such a non-validating parser will assume that the entity reference is defined in the external DTD subset it didn’t read; but rather than reporting the replacement text for that entity, it reports a skipped entity using the skippedEntity() callback method:

public void skippedEntity(String name)
    throws SAXException;

For example, according to the XHTML 1.0 specification, if a User Agent such as a browser “encounters an entity reference (other than one of the predefined HTML entities) for which the User Agent has processed no declaration (which could happen if the declaration is in the external subset which the User Agent hasn’t read), the entity reference should be processed as the characters (starting with the ampersand and ending with the semi-colon) that make up the entity reference.” In other words, rather than rendering &prescription_take; as the symbol ℞, the browser is supposed to draw it as simply &prescription_take;. If you were writing an XHTML browser that did not validate but did require full conformance to XHTML 1.0, you would probably implement the skippedEntity() method by passing an ampersand, the name of the entity reference, and a semicolon to the characters() method in the same content handler like this:

  public void skippedEntity(String name)
   throws SAXException {

    StringBuffer sb = new StringBuffer();
    sb.append('&');
    sb.append(name);
    sb.append(';');
    char[] text = new char[sb.length()];
    sb.getChars(0, sb.length(), text, 0)
    this.characters(text, 0, text.length);

  }

Skipped entities can also appear in attribute values. For example,

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
                    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <body>
     <div purpose="resum&eacute;">
     ...
     </div>
  </body>
</html>

This is one of the few holes in SAX. The parser will not report such an entity to you. The value it assigns to the attribute is calculated by simply deleting the entity reference. In this example, the value of the purpose attribute would be reported as “resum” if the parser does not read the DTD.

Prev | Up | Next
Ignorable White Space | Home | Receiving Locators
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified April 30, 2002


Receiving Locators
Prev Chapter 6. SAX Next

Receiving Locators

For debugging purposes, it’s often useful to know exactly where a particular element or other item appears. To provide this information, parsers should (but are not required to) implement the Locator interface. A Locator object knows at which point in which file the latest event was fired.

Example 6.15 summarizes the Locator interface. You see that it offers both public and system identifiers for the entity in which the start-tag/end-tag/processing instruction/skipped entity/etc. was found. Furthermore it tells you approximately at which line and column in that line the item begins. Lines and columns both begin at 1. If this information isn’t available for some reason (most commonly a well-formedness error very early in the document) these methods normally return -1.

Example 6.15. The SAX Locator interface

package org.xml.sax;

public interface Locator {

  public String getPublicId();
  public String getSystemId();
  public int    getLineNumber();
  public int    getColumnNumber();

}

The public identifier is normally a string like “-//OASIS//DTD DocBook XML V4.1.2//EN”. However, it’s often absent in which case getPublicId() returns null instead. The system identifier is almost always an absolute or relative URL if it’s known at all. However in some contexts no URL may be available. For instance, the XMLReader may be receiving its events from a JDOM SAXOutputter object or a stream of unknown origin rather than directly from a file or a network connection. In this case, getSystemId() may return null too. Since XML documents can be divided across multiple files, it’s possible that different items may appear at different URLs, though well-formedness guarantees that every start-tag has the same URL as its corresponding end-tag.

If the parser does provide location information, then it will invoke the ContentHandler’s setDocumentLocator() method before calling startDocument(). If you want location information to be available later, then you need to store a reference to this object somewhere, typically in a field. For example,

  private Locator locator;

  public void setDocumentLocator(Locator locator) {
    this.locator = locator;
  }

If the ContentHandler is used to parse multiple documents, you’ll receive a new Locator object for each document. That Locator is only good while its document is being parsed. Once the endDocument() method has been invoked, the Locator may not return sensible results.

Example 6.16 demoes this interface with a program that prints out line and column numbers for each method invocation. To run this program you need a parser that does provide location information; but most, including Xerces, do provide this.

Example 6.16. Determining the locations of events

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;

public class LocatorDemo implements ContentHandler {

  private Locator locator;

  public void setDocumentLocator(Locator locator) {
    this.locator = locator;
  }

  private void printLocation(String s) {

    int line = locator.getLineNumber();
    int column = locator.getColumnNumber();
    System.out.println(
     s + " at line " + line + "; column " + column
    );

  }

  public void startDocument() {
    printLocation("startDocument()");
  }

  public void endDocument() {
    printLocation("endDocument()");
  }

  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) {
    printLocation("startElement()");
  }

  public void endElement(String namespaceURI, String localName,
  String qualifiedName) {
    printLocation("endElement()");
  }

  public void characters(char[] text, int start, int length) {
    printLocation("characters()");
  }

  public void startPrefixMapping(String prefix, String uri) {
    printLocation("startPrefixMapping()");
  }

  public void endPrefixMapping(String prefix) {
    printLocation("endPrefixMapping()");
  }

  public void ignorableWhitespace(char[] text, int start,
   int length) {
    printLocation("ignorableWhitespace()");
  }

  public void processingInstruction(String target, String data) {
    printLocation("processingInstruction()");
  }

  public void skippedEntity(String name) {
    printLocation("skippedEntity()");
  }

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println("Usage: java SAXSpider URL1");
    }
    String uri = args[0];

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader();

      // Install the ContentHandler
      ContentHandler handler = new LocatorDemo();
      parser.setContentHandler(handler);
      parser.parse(uri);

    }
    catch (Exception e) {
      System.err.println(e);
    }

  } // end main

} // end LocatorDemo

Here’s the output when Example 6.13 is fed into this program:

C:\XMLJAVA>java LocatorDemo SymbolLookup.xml
startDocument() at line 1; column 1
startElement() at line 10; column 13
ignorableWhitespace() at line 11; column 3
startElement() at line 11; column 15
characters() at line 11; column 27
endElement() at line 11; column 41
ignorableWhitespace() at line 12; column 3
startElement() at line 12; column 11
ignorableWhitespace() at line 13; column 5
startElement() at line 13; column 12
ignorableWhitespace() at line 14; column 7
startElement() at line 14; column 14
ignorableWhitespace() at line 15; column 9
startElement() at line 15; column 17
characters() at line 17; column 9
endElement() at line 17; column 19
ignorableWhitespace() at line 18; column 7
endElement() at line 18; column 16
ignorableWhitespace() at line 19; column 5
endElement() at line 19; column 14
ignorableWhitespace() at line 20; column 3
endElement() at line 20; column 13
ignorableWhitespace() at line 21; column 1
endElement() at line 21; column 15
endDocument() at line -1; column -1

Prev | Up | Next
Receiving Skipped Entities | Home | What the ContentHandler Doesn’t Tell You
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified October 16, 2001


What the ContentHandler Doesn’t Tell You
Prev Chapter 6. SAX Next

What the ContentHandler Doesn’t Tell You

The ContentHandler interface is designed to provide everything most applications need to know about an XML instance document. The things it leaves out are things you rarely care about, though most of these are available through other callback interfaces discussed in upcoming chapters. These include:

  • Comments, unskipped entities, and CDATA sections, all of which are available through the LexicalHandler interface

  • The names, public IDs, system IDs and notations for unparsed entities and the names, public IDs, and system IDs for notations, all of which are available through the DTDHandler interface

  • ELEMENT, ATTLIST, and parsed ENTITY declarations from the DTD, all of which are reported through the DeclHandler interface

  • Validity errors and other non-fatal errors which are reported through the ErrorHandler interface

The only things that are truly not available in SAX2, even after all optional extensions are included, are:

  • The version, encoding, and standalone attributes from the XML declaration (scheduled to be added in SAX 2.1)

  • Insignificant white space in tags and before and after the root element

  • The order of attributes

  • The type of quotes that surround attributes

  • Character references

  • Pre-normalized attribute values

  • Whether an attribute was specified in the instance document or defaulted in from the DTD or schema

  • Whether empty elements are represented as <name></name> or <name />

  • Skipped entities in attribute values

The only common use case for most of this information is an XML editor. Editors are actually quite strange beasts compared to most client applications and really require a custom parser and API. None of the standard APIs or parsers provide all the information an editor needs.

Prev | Up | Next
Receiving Locators | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified October 16, 2001


Summary
Prev Chapter 6. SAX Next

Summary

SAX, the Simple API for XML, is a Java native, parser independent, event-based API for processing XML documents, suitable for use with very large documents and streaming data. SAX is mostly organized around interfaces. The implementation classes vary from one parser to the next.

The two key SAX interfaces are XMLReader, which represents the parser, and ContentHandler, through which the parser communicates with the client application. The client application registers its specific instance of the ContentHandler interface with an XMLReader through the setContentHandler() method. Then, as the XMLReader parses documents, it calls back to the methods in the ContentHandler object.

Only in the simplest programs do these callback methods give you all the information you want when you want it. Most of the time you have to gradually fill one or more data structures as the document is parsed. The complexity of any SAX program is largely a function of the complexity of the data structures you need to build.

Prev | Up | Next
What the ContentHandler Doesn’t Tell You | Home | Chapter 7. The XMLReader Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified October 15, 2001


Chapter 7. The XMLReader Interface
Prev Next

Chapter 7. The XMLReader Interface

Table of Contents

Building Parser Objects
Input
InputSource
EntityResolver
Exceptions and Errors
SAXExceptions
The ErrorHandler interface
Features and Properties
Getting and Setting Features
Getting and Setting Properties
Required Features
Standard Features
Standard Properties
Xerces Custom Features
Xerces Custom Properties
DTDHandler
Summary

The XML specification grants parsers a sometimes confusing amount of leeway in processing XML documents. Parsers are allowed to validate or not, resolve external entities or not, treat non-deterministic content models as errors or not, support non-standard encodings or not, check for namespace well-formedness or not, and much more. Depending on exactly which choices two parsers make for all these options, they can actually produce quite different pictures of the same XML document. Indeed, in a few cases one parser may even report a document to be well-formed while another reports that the document is malformed.

To support the wide range of capabilities of different parsers, the XMLReader interface that represents parsers in SAX is quite deliberately non-specific. It can be instantiated in a variety of different ways. It can read XML documents stored in a variety of media. It can be configured with features and properties both known and unknown. This chapter explores in detail the configuration and use of XMLReader objects.

Building Parser Objects

Since XMLReader is an interface, it has no constructors. Instead you use the static factory method XMLReaderFactory.createXMLReader() to retrieve an instance of XMLReader. In fact, there are two such methods in the XMLReaderFactory class:

public static XMLReader createXMLReader()
    throws SAXException;
public static XMLReader createXMLReader(String className)
    throws SAXException;

The first one returns the default XMLReader implementation class. This is specified by the org.xml.sax.driver Java system property. Parser vendors are supposed to modify this method to return an instance of their own parser in the event that this property is not set, though in practice few do this. Consequently when running a program that relies on XMLReaderFactory.createXMLReader() you may want to set the org.xml.sax.driver Java system property at the command line using the -D flag to the interpreter like this:

% java -Dorg.xml.sax.driver=com.fully.package.qualified.ParserClass
 MainClass

If there are multiple versions of the SAX classes in your class path, then whichever one the virtual machine finds first gets to choose which XMLReader implementation class to give you. However, if you know you want a specific class (e.g. org.apache.xerces.parsers.SAXParser) then you can ask for it by fully package-qualified name using the second XMLReaderFactory.createXMLReader() method. For example, this code asks for the Xerces parser by name:

XMLReader parser = XMLReaderFactory.createXMLReader(
 "org.apache.xerces.parsers.SAXParser"
);

If the class you request can’t be located, createXMLReader() throws a SAXException. Since there’s no guarantee that any particular parser is installed on any given system where your code may run, you should be prepared to catch and respond to this. Normally the correct response is to fall back to the default parser, like this:

XMLReader parser;
  try {
    parser = XMLReaderFactory.createXMLReader(
     "org.apache.xerces.parsers.SAXParser"
    );
  }
  catch (SAXException e) {
    try {
      parser = XMLReaderFactory.createXMLReader();
    }
    catch (SAXException e) {
      throw new NoClassDefFoundError("No SAX parser is available");
      // or whatever exception your method is declared to throw
    }
  }

Alternately, you can try multiple known parser classes until you find one that’s available. This code searches for several of the major parsers in my personal order of preference, only falling back on the default parser if none of these can be found:

  XMLReader parser;

  try { // Xerces
    parser = XMLReaderFactory.createXMLReader(
     "org.apache.xerces.parsers.SAXParser"
    );
  }
  catch (SAXException e1) {
    try { // Crimson
      parser = XMLReaderFactory.createXMLReader(
       "org.apache.crimson.parser.XMLReaderImpl"
      );
    }
    catch (SAXException e2) {
      try { // Ælfred
        parser = XMLReaderFactory.createXMLReader(
         "gnu.xml.aelfred2.XmlReader"
        );
      }
      catch (SAXException e3) {
        try { // Piccolo
          parser = XMLReaderFactory.createXMLReader(
            "com.bluecast.xml.Piccolo"
          );
        }
        catch (SAXException e4) {
          try { // Oracle
            parser = XMLReaderFactory.createXMLReader(
              "oracle.xml.parser.v2.SAXParser"
            );
          }
          catch (SAXException e5) {
            try { // default
              parser = XMLReaderFactory.createXMLReader();
            }
            catch (SAXException e6) {
              throw new NoClassDefFoundError(
                "No SAX parser is available");
                // or whatever exception your method is
                // declared to throw
            }
          }
        }
      }
    }
  }

I use this technique in my working code; and you’re more than welcome to copy it. However, because it’s quite long and repetitive, I’ll mostly stick to one named parser and a fallback to the default in the examples in this book.

I also occasionally see programs that use a constructor to retrieve an instance of a particular class. For example,

XMLReader parser = new SAXParser()

Or, worse yet,

SAXParser parser = new SAXParser()

This doesn’t let you do anything you can’t do with XMLReaderFactory.createXMLReader(). However, it does tie your code tightly to one particular parser and makes it a little more difficult to change parsers at a later date. At an absolute minimum, swapping in a different parser will require an edit and a recompile. However, if you use XMLReaderFactory.createXMLReader() instead, you can change parsers without even having access to the source code.

Prev | Up | Next
Summary | Home | Input
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified November 11, 2001


Input
Prev Chapter 7. The XMLReader Interface Next

Input

Once you’ve got an instance of XMLReader, you’re going to want to parse some documents with it. There are two methods that do this:

public void parse(String systemID)
    throws SAXException;
public void parse(InputSource in)
    throws SAXException;

In the last chapter you learned how these methods call back to the client application. What I want to look at now is how the document is fed into the parser.

The parse() method takes either a String containing a system ID or an InputSource object as an argument. An InputSource is a wrapper for the various kinds of input streams and readers from which an XML document can be read. If a system ID is passed to the parse() method, then it’s used to construct a new InputSource object which can be passed to the other overloaded parse() method.

The system ID is an absolute URL such as http://www.example.com/example.xml or a relative URL such as example.xml. Although XML allows any type of URI to be used for system IDs, the parser will need to resolve it. This means you need to use a URL here, not a URN; and that URL must have a scheme supported by your virtual machine’s protocol handlers. http and ftp are generally safe choices. [1]

Relative URLs are normally relative to the current working directory of the Java program. I’ve occasionally had problems resolving relative URLs on Windows when the URLs point into the local file system but outside the current working directory and its descendants. Most of the time, though, this is not a problem. Complicated relative URLs seem totally reliable on Unix and on web servers. More than anything else, this reflects the limitations of the java.net.URL class and the unfamiliarity with Windows of the programmers who wrote the Java class library.

InputSource

The InputSource class shown in Example 7.1 is an unusual three-way wrapper around an InputStream and/or a Reader and/or a system ID string. An XML document can be read from this source. Either the InputStream, the Reader, or the system ID can be set in the constructor. There’s also a no-args constructor that sets none of these. Those properties that aren’t set in the constructor can be set later with various setter methods. An encoding property can be set via a setter method but not in a constructor. Finally, the current value of all four properties can be retrieved using getter methods. Any properties that are not explicitly set will be null.

Example 7.1. The SAX InputSource class

package org.xml.sax;

public class InputSource {

  public InputSource()
  public InputSource(String systemID)
  public InputSource(InputStream byteStream)
  public InputSource(Reader characterStream)

  public void        setPublicId(String publicID)
  public String      getPublicId()
  public void        setSystemId(String systemID)
  public String      getSystemId()
  public void        setByteStream(InputStream byteStream)
  public InputStream getByteStream()
  public void        setEncoding(String encoding)
  public String      getEncoding()
  public void        setCharacterStream(Reader characterStream)
  public Reader      getCharacterStream()

}

What’s strange about this class is that there’s no guarantee that the three properties of each InputSource object are in any way related to each other. The byte stream might read one XML document, the character stream a different document, and the system ID might point to still a third document or no document at all. When the parser attempts to parse an InputSource object, it first tries to read from the reader, ignoring the encoding. If no reader is available, the parser then tries to read the document from the InputStream using the specified encoding. If no encoding has been specified, the parser will attempt to determine it from the XML document itself by using the first few bytes of the file and the encoding declaration. If neither a Reader nor an InputStream is available, then the parser will try to open a connection to the URI identified by the system ID. If all three options fail, then the parse() method throws a SAXException.

Note

Do not make both the byte stream and the character stream non-null in an InputSource object. The parser will not use the additional source as a backup for an initial source that has a problem (e.g. a system ID that points to a 404 Not Found error). It attempts to load the document from the first non-null property. If that fails, it throws an IOException or a SAXException. It does not consider any extra sources that might be available.

You should always set the system ID for an InputSource, even if you intend to read the actual data from a byte stream or a character stream. The system ID is the base URL for relative URLs found in the DOCTYPE declaration and external entity references. Some applications may also need it to resolve relative URLs in XML content such as XLink and XInclude elements. Relative URLs can't be resolved against an InputStream or Reader without the additional hint provided by the system ID. Finally, the system ID is used by the Locator and ErrorHandler interfaces to identify the file in which a particular element or problem appeared. This is especially important when you're working with XML documents (like the source code for this book, for one example) that are composed of many different files.

Most of the time a system ID is sufficient. However, the InputSource class can be quite useful when you can get a stream from a document but don’t have a convenient URL for it. For example, one common criticism of XML is that it’s verbose, that XML documents are too large compared to binary equivalents. This complaint is fallacious. In practice, most binary documents are bigger than XML equivalents. Nonetheless, this hasn’t stopped uninformed developers from complaining about XML’s verbosity. However, it’s quite straightforward to compress XML documents and decompress them as necessary, especially in Java where the java.util.zip package does all the hard work for you. The one downside to this approach is that URLs that point to your files no longer identify well-formed XML documents. Instead they point to a non-well-formed binary document. Thus they can’t be used as system IDs. However, you can still get an InputStream from these compressed files; and you can use that InputStream to create an InputSource. Then you parse the InputSource. For example, the following code parses the gzipped document found at http://www.example.com/bigdata.xml.gz:

  URL u = new URL("http://www.example.com/bigdata.xml.gz");
  InputStream raw = u.openStream();
  InputStream decompressed = new GZIPInputStream(raw);
  InputSource in = new InputSource(decompressed);
  in.setSystemId("http://www.example.com/bigdata.xml");
  parser.parse(in);

Similar techniques apply anytime you’ve got an InputStream that doesn’t come from a URL. If an XML document is stored in a String, you can use a StringReader to construct an InputSource. If the XML document is stored in a BLOB field in a relational database, you can use JDBC to retrieve a java.sql.Blob object, then use that class’s getBinaryStream() method to convert the BLOB into an InputStream from which an InputSource. can be constructed. If a UDP packet received by the DatagramSocket class contains an XML document, you can extract the data from the packet as a byte array using the DatagramPacket class’s getData() method, construct a ByteArrayInputStream from that array, and use the ByteArrayInputStream to construct an InputSource object for parsing.

EntityResolver

An XML document is made up of entities. Each entity is identified by public identifiers and/or system identifiers. The system IDs tend to be URLs. The public IDs generally require some sort of catalog system that can convert them into URLs. An XML parser reads each entity using an InputSource connected to the right URL. Most of the time you just give the parser a system ID or an InputSource pointing to the document entity, and let the parser figure out where to find any further entities referenced from the document entity. However, sometimes you want the parser to read from different URLs than the ones the document specifies. For example, the parser might ask for the XHTML DTD from the W3C web site. However, you might choose to replace that with a cached copy stored locally. Or perhaps the parser might ask for the SMIL 1.0 DTD, but you want to give it the SMIL 2.0 DTD instead.

The EntityResolver interface allows you to filter the parser’s requests for external parsed entities so you can replace the files it requests with your own copies, either faithful or modified. You might even use this interface to provide some form of custom proxy server support, though chances are that would be better implemented at the socket level rather than in the parsing API.

EntityResolver is a callback interface much like ContentHandler. It is attached to an XMLReader with set and get methods:

public void setEntityResolver(EntityResolver resolver);
public EntityResolver getEntityResolver();

The EntityResolver interface, summarized in Example 7.2, contains just a single method, resolveEntity(). If you register an EntityResolver with an XMLReader, then every time that XMLReader needs to load an external parsed entity, it will pass the entity’s public ID and system ID to resolveEntity() first. resolveEntity() can either return an InputSource or null. If it returns an InputSource, then this InputSource provides the entity’s replacement text. If it returns null, then the parser reads the entity in the same way it would have if there wasn’t an EntityResolver, probably just by using the system ID and the java.net.URL class.

Example 7.2. The EntityResolver interface

package org.xml.sax;

public interface EntityResolver {

  public InputSource resolveEntity(String publicId,
   String systemId) throws SAXException, IOException;

}

Example 7.3 is a simple EntityResolver implementation that maps the XHTML public IDs to URLs on a local server at http://www.cafeconleche.org. A more extensible implementation would allow the lists of IDs and URLs to be customized.

Example 7.3. An XHTML EntityResolver

import org.xml.sax.*;
import java.util.Hashtable;

public class LocalXHTML implements EntityResolver {

  private Hashtable entities = new Hashtable();

  // fill the list of URLs
  public LocalXHTML() {

    // The XHTML 1.0 DTDs
    this.addMapping("-//W3C//DTD XHTML 1.0 Strict//EN",
     "http://www.cafeconleche.org/DTD/xhtml1-strict.dtd");
    this.addMapping("-//W3C//DTD XHTML 1.0 Transitional//EN",
     "http://www.cafeconleche.org/DTD/xhtml1-transitional.dtd");
    this.addMapping("-//W3C//DTD XHTML 1.0 Frameset//EN",
     "http://www.cafeconleche.org/DTD/xhtml1-frameset.dtd");

    // The XHTML 1.0 entity sets
    this.addMapping("-//W3C//ENTITIES Latin 1 for XHTML//EN",
     "http://www.cafeconleche.org/DTD/xhtml-lat1.ent");
    this.addMapping("-//W3C//ENTITIES Symbols for XHTML//EN",
     "http://www.cafeconleche.org/DTD/xhtml-symbol.ent");
    this.addMapping("-//W3C//ENTITIES Special for XHTML//EN",
     "http://www.cafeconleche.org/DTD/xhtml-special.ent");

  }

  private void addMapping(String publicID, String URL) {
    entities.put(publicID, URL);
  }

  public InputSource resolveEntity(String publicID,
   String systemID) throws SAXException {

    if (entities.contains(publicID)) {
      String url = (String) entities.get(publicID);
      InputSource local = new InputSource(url);
      return local;
    }
    else return null;

  }

}

Other schemes are certainly possible. For example, instead of looking at the public ID, you could replace the host in the system ID to load the DTDs from a mirror site. You could bundle the DTDs into your application’s JAR file and load them from there. You could even hardwire the DTDs in the EntityResolver as string literals and load them with a StringReader.

[1] Technically, I suppose it’s not absolutely necessary that a parser use the java.net.URL class and protocol handlers to download content from a URL. However, in practice all Java parsers I’m aware of are implemented this way.

Prev | Up | Next
Chapter 7. The XMLReader Interface | Home | Exceptions and Errors
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


Exceptions and Errors
Prev Chapter 7. The XMLReader Interface Next

Exceptions and Errors

The XML specification defines three classes of problems that can occur in an XML document. In order of decreasing severity, these are

Fatal error

A well-formedness error. As soon as the parser detects it, it must throw in the towel and stop parsing. The parse() method throws a SAXParseException when a fatal error is detected. Parsers have a little leeway in whether they detect fatal errors. In particular non-validating parsers may not catch certain fatal errors that occur in the external DTD subset, and many parsers don’t actually check everything they’re supposed to check. However, if a parser does detect a fatal error, then it must give up and stop parsing.

Error

An error but not a well-formedness error. The most common such error is a validity error, though there are a few other kinds as well. Some parsers classify violations of namespace well-formedness as errors. Parsers may or may not detect these errors. If a parser does detect it, it may or may not throw a SAXParseException and it may or may not continue parsing. (Validity errors generally do not cause SAXParseExceptions. Other kinds of errors may, depending on the parser.) These sorts of errors are a source of some interoperability problems in XML because two parsers may behave differently given the same document.

Warning

Not itself an error. However, it may nonetheless indicate a mistake of some kind in the document. For example, a parser might issue a warning if it encountered an element named XMLDocument because all names beginning with “XML” (in any arrangement of case) are reserved by the W3C for future standards. Parsers may or may not detect problems like this. If a parser does detect one, it will not throw an exception and it will continue parsing.

In addition, a parser may encounter an I/O problem that has nothing to do with XML. For example, your cat might knock the Ethernet cable out of the back of your PC while you’re downloading a large XML document from a remote web server.

If the parser detects a well-formedness error in the document it’s parsing, parse() throws a SAXException. In the event of an I/O error, it throws an IOException. The parser may or may not throw a SAXException in the event of a non-fatal error, and will not throw an exception for a warning.

As you can see, the only kind of problem the parser is guaranteed to tell you about through an exception is the well-formedness error. If you want to be informed of the other kinds of errors and possible problems, you have to implement the ErrorHandler interface, and register your ErrorHandler implementation with the XMLReader.

SAXExceptions

The SAXException class, Example 7.4, is the generic exception class for almost anything other than an I/O problem that can go wrong while processing an XML document with SAX. Not only the parse() method but most of the callback methods in the various SAX interfaces are declared to throw this exception. If you detect a problem while processing an XML document, your code can throw its own SAXException.

Example 7.4. The SAXException class

package org.xml.sax;

public class SAXException extends Exception {

  public SAXException()
  public SAXException(String message)
  public SAXException(Exception rootCause)
  public SAXException(String message, Exception e)

  public String    getMessage()
  public Exception getException()
  public String    toString()

}

Nested Exceptions

SAXException may not always be the exception you want to throw, however. For example, suppose you’re parsing a document containing an XML digital signature, and the endElement() method notices that the Base-64 encoded text provided in the P element, which represents the prime modulus of a DSA key, does not decode to a prime number like it’s supposed to. You naturally want to throw a java.security.InvalidKeyException to warn the client application of this. However, endElement() cannot throw a java.security.InvalidKeyException, only a SAXException In this case, you wrap the exception you really want to throw inside a SAXException and throw the SAXException instead. For example,

Exception nestedException
 = new InvalidKeyException("Modulus is not prime!");
SAXException e = new SAXException(nestedException);
throw e;

The code that catches the SAXException can retrieve the original exception using the getException() method. For example, the client application method might indeed be declared to throw an InvalidKeyException so you could cast the nested exception to its real type and throw it into the appropriate catch block elsewhere in the call chain.

catch (SAXException e) {
  Exception rootCause = e.getException();
  if (rootCause == null) {
    // handle it as an XML problem...
  }
  else {
    if (rootCause instanceof InvalidKeyException) {
      InvalidKeyException ike = (InvalidKeyException) rootCause;
      throw ike;
    }
    else if (rootCause instanceof SomeOtherException) {
      SomeOtherException soe = (SomeOtherException) rootCause;
      throw soe;
    }
    …
  }
}

SAXException Subclasses

SAX defines several more specific subclasses of SAXException for specific problems, though most methods are only declared to throw a generic SAXException. These subclasses include SAXParseException, SAXNotRecognizedException, and SAXNotSupportedException. In addition parsers can extend SAXException with their own custom subclasses, though few do this.

A SAXParseException indicates a fatal error, error, or warning in an XML document. The parse() method of the XMLReader interface throws this when it encounters a well-formedness error. SAXParseException is also passed as an argument to the methods of the ErrorHandler interface to signal any of the three kinds of problems an XML document may contain.

Besides the usual exception methods like getMessage() and printStackTrace() that SAXParseException inherits from its superclasses, it provides methods to get the public ID and system ID of the file where the well-formedness error occurs (Remember, XML documents that use external parsed entities can be broken up over multiple separate files.) and the line number and column number where the well-formedness error appears within that file.

Example 7.5. The SAXParseException class

package org.xml.sax;

public class SAXParseException extends SAXException {

  public SAXParseException(String message, Locator locator)
  public SAXParseException(String message, Locator locator,
     Exception e)
  public SAXParseException(String message, String publicID,
   String systemID, int lineNumber, int columnNumber)
  public SAXParseException(String message, String publicID,
   String systemID, int lineNumber, int columnNumber,
   Exception e)

  public String getPublicId()
  public String getSystemId()
  public int    getLineNumber()
  public int    getColumnNumber()

}

The lines and column numbers reported by the parser for the problem may not always be perfectly accurate. Nonetheless, they should be close to where the problem begins or ends. (Some parsers give the line and column numbers for the start-tag of a problem element. Others give the give the line and column numbers for the end-tag.) If the document is so malformed that the parser can’t even begin working with it, especially if it isn’t an XML document at all, then the parser will probably indicate that the error occurred at line -1, column -1.

Example 7.6 enhances last chapter’s SAXChecker program so that it reports the line numbers of any well-formedness errors. Since there are two catch blocks, one for SAXParseException and one for the more generic SAXException, it’s possible to distinguish between well-formedness errors and other problems such as not being able to find the right XMLReader implementation class.

Example 7.6. A SAX program that parses a document and identifies the line numbers of any well-formedness errors

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.IOException;

public class BetterSAXChecker {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java BetterSAXChecker URL");
      return;
    }
    String document = args[0];

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader();
      parser.parse(document);
      System.out.println(document + " is well-formed.");
    }
    catch (SAXParseException e) {
      System.out.print(document + " is not well-formed at ");
      System.out.print("Line " + e.getLineNumber()
       + ", column " +  e.getColumnNumber() );
      System.out.println(" in the entity " + e.getSystemId());
    }
    catch (SAXException e) {
      System.out.println("Could not check document because "
       + e.getMessage());
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }

  }

}

Here’s the output I got when I first ran it across my Cafe con Leche home page. The first time I did not specify a parser which produced a generic SAXException. The second time I corrected that mistake, and a SAXParseException signaled a well-formedness error.

%java BetterSAXChecker http://www.cafeconleche.org
Could not check document because System property
 org.xml.sax.driver not specified
%java -Dorg.xml.sax.driver=org.apache.xerces.parsers.SAXParser
 BetterSAXChecker http://www.cafeconleche.org
http://www.cafeconleche.org is not well-formed at Line 64,
column 64 in the entity http://www.cafeconleche.org/

Not necessarily fatal errors

XML includes a few errors that fall into a gray area. These are errors but neither fatal well-formedness errors nor non-fatal validity errors. The most common such error is an ambiguous content model in an element declaration. For example, consider this declaration which states that an Actor can have between 0 and 2 Parts:

<!ELEMENT Actor (Part?, Part?)>

The problem occurs with an Actor element that has one Part like this:

<Actor>
  <Part>Cyrano</Part>
</Actor>

Does this one Part match the first Part in the content model or the second? There’s no way to tell. Some parsers have trouble with this construct while other parsers don’t notice any problem at all. The XML specification says this is an error, but does not classify it as a fatal error.

Different parsers treat these not necessarily fatal errors differently. Some parsers throw a SAXParseException when one is encountered. Other parsers let them pass without comment. And still others report them in a different way but not throw an exception. For maximum compatibility, try to design your DTDs and instance documents so these are not a problem.

The ErrorHandler interface

Throwing an exception aborts the parsing process. Not all problems encountered in an XML document necessarily require such a radical step. In particular, validity errors are not signaled by an exception because that would stop parsing. If you want your program to be informed of non-fatal errors, you must register an ErrorHandler object with the XMLReader. Then the parser will tell you about problems in the document by passing (not throwing!) a SAXParseException to one of the methods in this object.

Example 7.7 summarizes the ErrorHandler interface. As you can see it has three callback methods corresponding to the three different kinds of problems a parser may detect. When the parser detects one of these problems, it passes a SAXParseException to the appropriate method. If you want to treat errors or warnings as fatal, then you can throw the exception you were passed. (The parse() method will always throw an exception for a fatal error, even if you don’t.) If you don’t want to treat them as fatal (and mostly you don’t), then you can do something else with the information wrapped in the exception.

Example 7.7. The ErrorHandler interface

package org.xml.sax;

public interface ErrorHandler {

  public void warning(SAXParseException exception)
   throws SAXException;
  public void error(SAXParseException exception)
   throws SAXException;
  public void fatalError(SAXParseException exception)
   throws SAXException;

}

These two methods install an ErrorHandler into an XMLReader:

public void setErrorHandler(ErrorHandler handler);
public ErrorHandler getErrorHandler();

You can uninstall an ErrorHandler by passing null to setErrorHandler().

Example 7.8 is a program that checks documents for well-formedness errors and other problems. All errors detected are reported, no matter how small, through the ErrorHandler interface.

Example 7.8. A SAX program that reports all problems found in an XML document

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.IOException;

public class BestSAXChecker implements ErrorHandler {

  public void warning(SAXParseException exception) {

    System.out.println("Warning: " + exception.getMessage());
    System.out.println(" at line " + exception.getLineNumber()
     + ", column " + exception.getColumnNumber());
    System.out.println(" in entity " + exception.getSystemId());

  }

  public void error(SAXParseException exception) {

    System.out.println("Error: " + exception.getMessage());
    System.out.println(" at line " + exception.getLineNumber()
     + ", column " + exception.getColumnNumber());
    System.out.println(" in entity " + exception.getSystemId());

  }

  public void fatalError(SAXParseException exception) {

    System.out.println("Fatal Error: " + exception.getMessage());
    System.out.println(" at line " + exception.getLineNumber()
     + ", column " + exception.getColumnNumber());
    System.out.println(" in entity " + exception.getSystemId());

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java BestSAXChecker URL");
      return;
    }
    String document = args[0];

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader();
      ErrorHandler handler = new BestSAXChecker();
      parser.setErrorHandler(handler);
      parser.parse(document);
      // If the document isn't well-formed, an exception has
      // already been thrown and this has been skipped.
      System.out.println(document + " is well-formed.");
    }
    catch (SAXParseException e) {
      System.out.print(document + " is not well-formed at ");
      System.out.println("Line " + e.getLineNumber()
       + ", column " +  e.getColumnNumber() );
      System.out.println(" in entity " + e.getSystemId());
    }
    catch (SAXException e) {
      System.out.println("Could not check document because "
       + e.getMessage());
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }

  }

}

Here’s the output from running BestSAXChecker across the Docbook XML source code for an early version of this chapter:

%java -Dorg.xml.sax.driver=org.apache.xerces.parsers.SAXParser
 BestSAXChecker xmlreader.xml
Error: The namespace prefix "xinclude" was not declared.
 at line 349, column 92
 in entity file:///D:/books/XMLJAVA/xmlreader.xml
Error: The namespace prefix "xinclude" was not declared.
 at line 530, column 95
 in entity file:///D:/books/XMLJAVA/xmlreader.xml
Error: The namespace prefix "xinclude" was not declared.
 at line 545, column 84
 in entity file:///D:/books/XMLJAVA/xmlreader.xml
Error: The namespace prefix "xinclude" was not declared.
 at line 688, column 93
 in entity file:///D:/books/XMLJAVA/xmlreader.xml
Fatal Error: The element type "para" must be terminated by the
matching end-tag "</para>".
 at line 706, column 42
 in entity file:///D:/books/XMLJAVA/xmlreader.xml
Could not check document because Stopping after fatal error:
The element type "para" must be terminated by the matching
 end-tag "</para>".

BestSAXChecker complains several times about an undeclared namespace prefix for the XInclude elements I use to merge in source code examples like Example 7.8. Then about three quarters of the way through the document, it encounters a well-formedness error where I neglected to put an end-tag in the right place. At this point parsing stops. If there are any errors after that point, they aren’t reported. Once I fixed those problems, the file became well-formed and valid:

%java -Dorg.xml.sax.driver=org.apache.xerces.parsers.SAXParser
 BestSAXChecker xmlreader.xml
xmlreader.xml is well-formed.

Beyond simple well-formedness, which errors this program catches depends on the underlying parser. All conformant parsers detect all well-formedness errors. Most modern parsers should also catch any violations of namespace well-formedness. Whether this program catches validity errors depends on the parser. Most parsers do not validate by default. Instead they require the client application to explicitly request validation by setting the http://xml.org/sax/features/validation feature to true. I take this subject up next.

Prev | Up | Next
Input | Home | Features and Properties
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


Features and Properties
Prev Chapter 7. The XMLReader Interface Next

Features and Properties

SAX parsers, that is, XMLReader objects, are configured by setting features and properties. A feature has a boolean true/false value. A property has an object value. Both features and properties are named by absolute URIs. This allows just a handful of standard methods to support an arbitrary number of standard and non-standard features and properties of various types.

Features and properties can be read-only, write-only (rare), or read-write. If you attempt to change a read-only feature or property, a SAXNotSupportedException, a subclass of SAXException. is thrown. The accessibility of a feature or property can change depending on whether or not the XMLReader is currently parsing a document. For example, you can turn validation on or off before or after parsing a document, but not while the XMLReader is parsing a document.

Getting and Setting Features

The XMLReader interface provides these two methods to turn features on and off:

public void setFeature(String name, boolean value)
    throws SAXNotRecognizedException, SAXNotSupportedException;
public boolean getFeature(String name)
    throws SAXNotRecognizedException, SAXNotSupportedException;

The first argument is the name of the feature to set or get. Feature names are absolute URIs. Standard features that are supported by multiple parsers have names that begin with http://xml.org/sax/features/. For example, this next code fragment checks to see if the XMLReader object parser is currently validating; and, if it isn’t, turns on validation by setting the feature http://xml.org/sax/features/validation to true.

if (!parser.getFeature("http://xml.org/sax/features/validation")) {
  parser.setFeature("http://xml.org/sax/features/validation", true);
}

However, different parsers also support non-standard, custom features. The names of these features begin with URLs somewhere in the parser vendor’s domain. For example, non-standard features of the Xerces parser from the XML Apache Project begin with http://apache.org/xml/features/.

If the XMLReader object can never access the feature you’re trying to get or set, setFeature() throws a SAXNotRecognizedException. On the other hand, if you try to get or set a feature that the parser recognizes but cannot access at the current time, setFeature() throws a SAXNotSupportedException. Both are subclasses of SAXException. For example, if parser is a non-validating parser like gnu.xml.aelfred2.SAXDriver, then the above code would throw SAXNotRecognizedException. However, if parser is a validating parser like Xerces but the setFeature() method were invoked while it was parsing a document, then it would throw a SAXNotSupportedException because you can’t turn on validation halfway through a document. Since these are checked exceptions, you’ll need to either catch these exceptions or declare that your method throws them. For example,

try {
  if (!parser.getFeature("http://xml.org/sax/features/validation")) {
    parser.setFeature("http://xml.org/sax/features/validation", true);
  }
}
catch (SAXNotRecognizedException) {
  System.out.println(parser + " is not a validating parser.");
}
catch (SAXNotSupportedException) {
  System.out.println(
   "Cannot turn on validation right now. Try again later."
  );
}

Getting and Setting Properties

The XMLReader interface uses these two methods to set and get the values of properties:

public void setProperty(String name, Object value)
    throws SAXNotRecognizedException, SAXNotSupportedException;
public Object getProperty(String name)
    throws SAXNotRecognizedException, SAXNotSupportedException;

Properties are named by absolute URIs, just like features. Standard properties have names that begin with http://xml.org/sax/properties/ such as http://xml.org/sax/properties/declaration-handler and http://xml.org/sax/properties/xml-string. However, most parsers also support some non-standard, custom properties. The names of these will begin with URLs somewhere in the parser vendor’s domain. For example, non-standard properties of the Xerces parser from the XML Apache Project begin with http://apache.org/xml/properties/, for instance http://apache.org/xml/properties/schema/external-noNamespaceSchemaLocation.

The value of a property is an object. The type of the object depends on which property it is. For instance, the value of the http://xml.org/sax/properties/declaration-handler property is an org.xml.sax.ext.DeclHandler while the value of the http://xml.org/sax/properties/xml-string property is a java.lang.String. Passing an object of the wrong type for the property to setProperty() results in a SAXNotSupportedException.

For example, suppose you’re using Xerces and you want to set the schema location for elements that are not in any namespace to http://www.example.com/schema.xsd. This code fragment accomplishes that:

try {
  parser.setProperty("http://apache.org/xml/properties/schema/"
   + "external-noNamespaceSchemaLocation",
   "http://www.example.com/schema.xsd");
}
catch (SAXNotRecognizedException) {
  System.out.println(parser
   + " is not a schema-validating parser.");
}
catch (SAXNotSupportedException) {
  System.out.println(
   "Cannot change the schema right now. Try again later."
  );
}

Required Features

There are only a couple of features which all SAX parsers must support and no absolutely required properties. The two required features are:

  • http://xml.org/sax/features/namespaces

  • http://xml.org/sax/features/namespace-prefixes

The http://xml.org/sax/features/namespaces feature determines whether namespace URIs and local names are passed to startElement() and endElement(). The default, true, passes both namespace URIs and local names. However, if http://xml.org/sax/features/namespaces is false, then the parser may pass the namespace URI and the local name, or it may just pass empty strings for these two arguments. The default is true, and there’s not a lot of reason to change it. (You can always ignore the URI and local name if you don’t need them.)

The http://xml.org/sax/features/namespace-prefixes feature determines two things:

  • Whether or not namespace declaration xmlns and xmlns:prefix attributes are included in the Attributes list passed to startElement(). The default, false, is not to include them.

  • Whether or not the qualified names should be passed as the third argument to the startElement() method. The default, false, is, not to require qualified names. However, even if http://xml.org/sax/features/namespace-prefixes is false, parsers are allowed to report the qualified name, and most do so.

For example, consider this start-tag:

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:dc="http://www.purl.org/dc/" id="R1">

If http://xml.org/sax/features/namespace-prefixes is false and http://xml.org/sax/features/namespaces is true, then when a SAX parser reads this tag it may invoke the startElement() method in its registered ContentHandler object with these arguments:

startElement(
  namespaceURI="http://www.w3.org/1999/02/22-rdf-syntax-ns#",
  localName = "RDF",
  qualifiedName="",
  attributes={id="R1"}
)

Alternately, it can choose to provide the qualified name even though it isn’t required to:

startElement(
  namespaceURI="http://www.w3.org/1999/02/22-rdf-syntax-ns#",
  localName = "RDF",
  qualifiedName="rdf:RDF",
  attributes={id="R1"}
)

However, if http://xml.org/sax/features/namespace-prefixes is true and http://xml.org/sax/features/namespaces is true, then when a SAX parser reads this tag it invokes the startElement() method like this:

startElement(
  namespaceURI="http://www.w3.org/1999/02/22-rdf-syntax-ns#",
  localName = "RDF",
  qualifiedName="rdf:RDF",
  attributes={id="R1", xmlns:dc="http://www.purl.org/dc/",
   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"}
)

If http://xml.org/sax/features/namespace-prefixes is true and http://xml.org/sax/features/namespaces is false, then when a SAX parser reads this tag it may invoke the startElement() method like this:

startElement(
  namespaceURI="",
  localName = "",
  qualifiedName="rdf:RDF",
  attributes={id="R1", xmlns:dc="http://www.purl.org/dc/",
   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"}
)

Then again it may provide the namespace URI and local name anyway, even though it doesn’t have to:

startElement(
  namespaceURI="http://www.w3.org/1999/02/22-rdf-syntax-ns#",
  localName = "RDF",
  qualifiedName="rdf:RDF",
  attributes={id="R1", xmlns:dc="http://www.purl.org/dc/",
   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"}
)

In other words,

  • The parser is only guaranteed to provide the namespace URIs and local names of elements and attributes if http://xml.org/sax/features/namespaces is true (which it is by default).

  • The parser is only guaranteed to provide the qualified names of elements and attributes if http://xml.org/sax/features/namespace-prefixes is true (which it is not by default).

  • The parser provides namespace declaration attributes if and only if http://xml.org/sax/features/namespace-prefixes is true (which it is not by default).

  • The parser always has the option to provide the namespace URI, local name, and qualified name, regardless of the values of http://xml.org/sax/features/namespaces and http://xml.org/sax/features/namespace-prefixes. However, you should not rely on this behavior.

To summarize, the defaults are fine as long as you don’t care about namespace prefixes, only local names and URIs.

Standard Features

Besides the two required features, SAX defines a number of standard features which parsers may support if they choose. These have names which are consistent across different parsers and include:

  • http://xml.org/sax/features/external-general-entities

  • http://xml.org/sax/features/external-parameter-entities

  • http://xml.org/sax/features/string-interning

  • http://xml.org/sax/features/validation

external-general-entities

If http://xml.org/sax/features/external-general-entities is true, then the parser resolves all external general entity references. If it’s false, it does not. If the parser is validating, then this feature is required to be true.

The default is parser-dependent. Not all parsers are able to resolve external entity references. Attempting to set this to true with a parser that cannot resolve external entity references will throw a SAXNotRecognizedException.

external-parameter-entities

If http://xml.org/sax/features/external-parameter-entities is true, then the parser resolves all external parameter entity references. If it’s false, it does not. If the parser is validating, then this feature is required to be true.

The default is parser-dependent. Not all parsers are able to resolve external entity references. Attempting to set this to true with a parser that cannot resolve external entity references will throw a SAXNotRecognizedException.

string-interning

If http://xml.org/sax/features/string-interning is true, then the parser internalizes all XML names using the intern() method of the String class before passing them to the various callback methods. Thus if there are 100 different paragraph elements in your document, the parser will only use one "paragraph" string for all 100 start-tags and 100 end-tags rather than 200 separate strings. This can save memory as well as allowing you to compare element names using the == operator instead of the equals() method. Besides element names, this also affects attribute names, entity names, notation names, namespace prefixes, and namespace URIs. The default is parser-dependent.

validation

If http://xml.org/sax/features/validation is true, then the parser validates the document against its DTD. Of course not all parsers are capable of doing this. Attempting to set http://xml.org/sax/features/validation to true for a parser that doesn’t know how to validate will throw a SAXNotRecognizedException.

Since validation requires resolving all external entity references, setting http://xml.org/sax/features/validation to true automatically sets http://xml.org/sax/features/external-general-entities and http://xml.org/sax/features/external-parameter-entities to true as well.

The default value of this feature is allegedly parser-dependent, though I’ve yet to encounter a parser that turns it on by default.

Example 7.9 is a program that uses this feature to validate documents. As well as setting the http://xml.org/sax/features/validation feature to true, it’s also necessary to register an ErrorHandler object that can receive messages about validity errors.

Example 7.9. A SAX program that validates documents

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.IOException;

public class SAXValidator implements ErrorHandler {

  // Flag to check whether any errors have been spotted.
  private boolean valid = true;

  public boolean isValid() {
    return valid;
  }

  // If this handler is used to parse more than one document,
  // its initial state needs to be reset between parses.
  public void reset() {
    // Assume document is valid until proven otherwise
    valid = true;
  }

  public void warning(SAXParseException exception) {

    System.out.println("Warning: " + exception.getMessage());
    System.out.println(" at line " + exception.getLineNumber()
     + ", column " + exception.getColumnNumber());
    // Well-formedness is a prerequisite for validity
    valid = false;

  }

  public void error(SAXParseException exception) {

    System.out.println("Error: " + exception.getMessage());
    System.out.println(" at line " + exception.getLineNumber()
     + ", column " + exception.getColumnNumber());
    // Unfortunately there's no good way to distinguish between
    // validity errors and other kinds of non-fatal errors
    valid = false;

  }

  public void fatalError(SAXParseException exception) {

    System.out.println("Fatal Error: " + exception.getMessage());
    System.out.println(" at line " + exception.getLineNumber()
     + ", column " + exception.getColumnNumber());

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java SAXValidator URL");
      return;
    }
    String document = args[0];

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader();
      SAXValidator handler = new SAXValidator();
      parser.setErrorHandler(handler);
      // Turn on validation.
      parser.setFeature(
       "http://xml.org/sax/features/validation", true);
      parser.parse(document);
      if (handler.isValid()) {
        System.out.println(document + " is valid.");
      }
      else {
        // If the document isn't well-formed, an exception has
        // already been thrown and this has been skipped.
        System.out.println(document + " is well-formed.");
      }
    }
    catch (SAXParseException e) {
      System.out.print(document + " is not well-formed at ");
      System.out.println("Line " + e.getLineNumber()
       + ", column " +  e.getColumnNumber() );
    }
    catch (SAXException e) {
      System.out.println("Could not check document because "
       + e.getMessage());
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }

  }

}

Here’s the beginning of the output from running it across the Docbook XML source code for an early version of this chapter:

%java -Dorg.xml.sax.driver=org.apache.xerces.parsers.SAXParser SAXValidator
 xmlreader.xml
Error: Element type "xinclude:include" must be declared.
 at line 344, column 92
Error: Attribute "href" must be declared for
 element type "xinclude:include".
 at line 344, column 92
Error: Attribute "parse" must be declared for
 element type "xinclude:include".
 at line 344, column 92
Error: The content of element type "programlisting" must match
"(#PCDATA|footnoteref|xref|abbrev|acronym|citation|citerefentry
|citetitle|emphasis|firstterm|foreignphrase|glossterm|footnote
|phrase|quote|trademark|wordasword|link|olink|ulink|action
|application|classname|methodname|interfacename|exceptionname
|ooclass|oointerface|ooexception|command|computeroutput|database
|email|envar|errorcode|errorname|errortype|filename|function
|guibutton|guiicon|guilabel|guimenu|guimenuitem|guisubmenu
|hardware|interface|keycap|keycode|keycombo|keysym|literal
|constant|markup|medialabel|menuchoice|mousebutton|option
|optional|parameter|prompt|property|replaceable|returnvalue
|sgmltag|structfield|structname|symbol|systemitem|token|type
|userinput|varname|anchor|author|authorinitials|corpauthor
|modespec|othercredit|productname|productnumber|revhistory|remark
|subscript|superscript|inlinegraphic|inlinemediaobject
|inlineequation|synopsis|cmdsynopsis|funcsynopsis|classsynopsis
|fieldsynopsis|constructorsynopsis|destructorsynopsis
|methodsynopsis|indexterm|beginpage|co|lineannotation)*".
 at line 344, column 110
…
xmlreader.xml is well-formed.

SAXValidator is complaining about the XInclude elements I use to merge in source code examples like Example 7.9. These are not expected by the Docbook DTD. They need to be replaced before the file becomes valid. Once I do that, the merged file (ch07.xml) is valid:

%java -Dorg.xml.sax.driver=org.apache.xerces.parsers.SAXParser SAXValidator
 ch07.xml
ch07.xml is valid.

Standard Properties

SAX does not require parsers to support any properties. However, it does define four standard properties which parsers may support if they choose. These are:

  • http://xml.org/sax/properties/declaration-handler

  • http://xml.org/sax/properties/dom-node

  • http://xml.org/sax/properties/lexical-handler

  • http://xml.org/sax/properties/xml-string

xml-string

http://xml.org/sax/properties/xml-string is a read-only property that contains the string of text corresponding to the current SAX event. For example, in the startElement() method, this property would contain the actual start-tag that caused the method invocation.

This property can be used in a very straightforward program that echoes an XML document onto a Writer, as shown in Example 7.10. Assuming a validating parser, the parsing process merges a document that was originally split across multiple parsed entities into a single entity. Here each callback method in the ContentHandler simply invokes a private method that writes out the current value of the http://xml.org/sax/properties/xml-string property.

Example 7.10. A SAX program that echoes the parsed document

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.*;

public class DocumentMerger implements ContentHandler {

  private XMLReader parser;
  private Writer out;

  public DocumentMerger(XMLReader parser, Writer out) {
    this.parser = parser;
    this.out = out;
  }

  private void output() throws SAXException {

    try {
      String s = (String) parser.getProperty(
       "http://xml.org/sax/properties/xml-string");
      out.write(s);
    }
    catch (IOException e) {
      throw new SAXException("Nested IOException", e);
    }

  }

  public void setDocumentLocator(Locator locator) {}

  public void startDocument() throws SAXException {
    this.output();
  }
  public void endDocument() throws SAXException {
    this.output();
  }

  public void startPrefixMapping(String prefix, String uri)
   throws SAXException {
    this.output();
  }

  public void endPrefixMapping(String prefix)
   throws SAXException {
    this.output();
  }

  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {
    this.output();
  }

  public void endElement(String namespaceURI, String localName,
   String qualifiedName) throws SAXException {
    this.output();
  }

  public void characters(char[] text, int start, int length)
   throws SAXException {
    this.output();
  }

  public void ignorableWhitespace(char[] text, int start,
   int length) throws SAXException {
    this.output();
  }

  public void processingInstruction(String target, String data)
   throws SAXException {
    this.output();
  }

  public void skippedEntity(String name)
   throws SAXException {
    this.output();
  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println(
       "Usage: java DocumentMerger url"
      );
      return;
    }

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader();

      // Since this just writes onto the console, it's best
      // to use the system default encoding, which is what
      // we get by not specifying an explicit encoding here.
      Writer out = new OutputStreamWriter(System.out);
      ContentHandler handler = new DocumentMerger(parser, out);
      parser.setContentHandler(handler);

      parser.parse(args[0]);

      out.flush();
      out.close();
    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

The document that's output may not be quite the same as the document that was read. Character references will have been resolved. General entity references will probably have been resolved. Parts of the prolog, especially the DOCTYPE declaration, may be missing. Attributes that were read in from defaults in the DTD will be explicitly specified. However, the complete information content of the original document should be present, even if the form is different.

The biggest issue with this program is finding a parser that recognizes the http://xml.org/sax/properties/xml-string property. In my tests, Xerces 1.4.3, Crimson, and Ælfred all threw a SAXNotRecognizedException or a SAXNotSupportedException. I have not yet found a parser that supports this property, and there's some suspicion in the SAX community that defining it in the first place may have been` a mistake.

dom-node

The http://xml.org/sax/properties/dom-node property contains the org.w3c.dom.Node object corresponding to the current SAX event. For example, in the startElement() and endElement() methods, this property contains an org.w3c.dom.Element object representing that element. In the characters() method, this property contains the org.w3c.dom.Text object which contained the characters from which the text had been read.

lexical-handler

Lexical events are those ephemera of parsing that don’t really mean anything. In some sense, they really aren’t part of the document’s information. Comments are the most obvious example. However, lexical data also includes entity boundaries, CDATA section delimiters, and the DOCTYPE declaration. What unifies all these is that they really don’t matter 99.9% of the time. Unfortunately, there’s that annoying 0.1% when you really do care about some lexical detail you’d normally ignore.

Parsers are not required to report lexical data; but if they want to do so, SAX provides a standard callback interface they can use, LexicalHandler, shown in Example 7.11. However, this interface is optional. Parsers are not required to support it. Notice that it is in the org.xml.sax.ext package, not the core org.xml.sax package.

Example 7.11. The LexicalHandler interface

package org.xml.sax.ext;

public interface LexicalHandler {

  public void startDTD(String name, String publicId,
   String systemId) throws SAXException;
  public void endDTD() throws SAXException;

  public void startEntity(String name)
   throws SAXException;
  public void endEntity(String name) throws SAXException;

  public void startCDATA() throws SAXException;
  public void endCDATA() throws SAXException;

  public void comment(char[] text, int start, int length)
   throws SAXException;

}

Because parsers are not required to support the LexicalHandler interface, it can’t be registered with a setLexicalHandler() method in XMLReader like the other callback interfaces. Instead, it’s set as the value of the http://xml.org/sax/properties/lexical-handler property.

For example, Example 7.12 is a concrete implementation of LexicalHandler that dumps comments from an XML document onto System.out.

Example 7.12. An implementation of the LexicalHandler interface

import org.xml.sax.*;
import org.xml.sax.ext.LexicalHandler;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.IOException;

public class CommentReader implements LexicalHandler {

  public void comment (char[] text, int start, int length)
   throws SAXException {

    String comment = new String(text, start, length);
    System.out.println(comment);

  }

  public static void main(String[] args) {

    // set up the parser
    XMLReader parser;
    try {
      parser = XMLReaderFactory.createXMLReader();
    }
    catch (SAXException e) {
      System.err.println("Error: could not locate a parser.");
      System.err.println(
       "Try setting the org.xml.sax.driver system property to "
       + "the fully package qualified name of your parser class."
      );
      return;
    }

    // turn on comment handling
    try {
      LexicalHandler handler = new CommentReader();
      parser.setProperty(
       "http://xml.org/sax/properties/lexical-handler", handler);
    }
    catch (SAXNotRecognizedException e) {
      System.err.println(
       "Installed XML parser does not provide lexical events...");
      return;
    }
    catch (SAXNotSupportedException e) {
      System.err.println(
       "Cannot turn on comment processing here");
      return;
    }

    if (args.length == 0) {
      System.out.println("Usage: java CommentReader URL");
    }

    // start parsing...
    try {
      parser.parse(args[0]);
    }
    catch (SAXParseException e) { // well-formedness error
      System.out.println(args[0] + " is not well formed.");
      System.out.println(e.getMessage()
       + " at line " + e.getLineNumber()
       + ", column " + e.getColumnNumber());
    }
    catch (SAXException e) { // some other kind of error
      System.out.println(e.getMessage());
    }
    catch (IOException e) {
      System.out.println("Could not read " + args[0]
       + " because of the IOException " + e);
    }

  }

  // do-nothing methods not needed in this example
  public void startDTD(String name, String publicId,
   String systemId) throws SAXException {}
  public void endDTD() throws SAXException {}
  public void startEntity(String name) throws SAXException {}
  public void endEntity(String name) throws SAXException {}
  public void startCDATA() throws SAXException {}
  public void endCDATA() throws SAXException {}

}

The main() method builds an XMLReader, constructs an instance of CommentReader, and uses setFeature() to make this CommentReader the parser’s LexicalHandler. Then it parses the document indicated on the command line.

It’s amusing to run this across the XML source for various W3C specifications. For example, Here’s the output when the XML version of the XML 1.0 specification, second edition, is fed into CommentReader:

%java CommentReader http://www.w3.org/TR/2000/REC-xml-20001006.xml
ArborText, Inc., 1988-2000, v.4002
 ...............................................................
 XML specification DTD .........................................
 ...............................................................

TYPICAL INVOCATION:
#  <!DOCTYPE spec PUBLIC
#       "-//W3C//DTD Specification V2.1//EN"
#       "http://www.w3.org/XML/1998/06/xmlspec-v21.dtd">

PURPOSE:
  This XML DTD is for W3C specifications and other technical reports.
  It is based in part on the TEI Lite and Sweb DTDs.
…

The comments you’re seeing are actually from the DTD used by the XML specification. Comments and processing instructions in the DTD, both internal and external subsets, are reported to their respective callback methods, just like comments and processing instructions in the instance document.

Example 7.12 is a pure LexicalHandler that does not implement any of the other SAX callback interfaces like ContentHandler. However, it’s not uncommon to implement several callback interfaces in one class. Among other advantages, that makes it a lot easier to write programs that rely on information that’s available in different interfaces.

declaration-handler

The http://xml.org/sax/properties/declaration-handler property identifies the parser’s DeclHandler. DeclHandler, summarized in Example 7.13, is an optional interface in the org.xml.sax.ext package used by parsers to report those parts of the DTD that don’t affect the content of instance documents, specifically ELEMENT, ATTLIST and parsed ENTITY declarations. Together with the information reported by the DTDHandler, this gives you enough information to reproduce a parsed document’s DTD. The reproduced DTD may not be exactly the same as the original DTD. For instance, parameter entities will have been resolved and only the first declaration of each general entity will be reported. Nonetheless, the model represented by the entire DTD should be intact.

Example 7.13. The DeclHandler interface

package org.xml.sax.ext;

public interface DeclHandler {

  public void elementDecl(String name, String model)
   throws SAXException;
  public void attributeDecl(String elementName,
   String attributeName, String type, String mode,
   String defaultValue) throws SAXException;
  public void internalEntityDecl(String name, String value)
   throws SAXException;
  public void externalEntityDecl(String name, String publicID,
   String systemID) throws SAXException;

}

Example 7.14 is a little DeclHandler I whipped up to help me make sense out of heavily modular, very customizable DTDs like XHTML 1.1 or SMIL 2.0. It takes advantage of the fact that all parameter entity references and conditional sections are replaced before the methods of DeclHandler are called. It implements the DeclHandler interface with methods that copy each declaration onto System.out. However, because parameter entity references and conditional sections are resolved before these methods are invoked, it outputs a single monolithic DTD. I can see, for example, exactly what the content model for an element such as blockquote really is without having to manually trace the parameter entity references through seven separate modules, and figuring out which modules are likely to be included and which ignored.

Example 7.14. A program that prints out a complete DTD

import org.xml.sax.*;
import org.xml.sax.ext.DeclHandler;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.IOException;

public class DTDMerger implements DeclHandler {

  public void elementDecl(String name, String model)
   throws SAXException {
    System.out.println("<!ELEMENT " + name + " " + model + " >");
  }

  public void attributeDecl(String elementName,
   String attributeName, String type, String mode,
   String defaultValue) throws SAXException {

    System.out.print("<!ATTLIST ");
    System.out.print(elementName);
    System.out.print(" ");
    System.out.print(attributeName);
    System.out.print(" ");
    System.out.print(type);
    System.out.print(" ");
    if (mode != null) {
      System.out.print(mode + " ");
    }
    if (defaultValue != null) {
      System.out.print('"' + defaultValue + "\" ");
    }
    System.out.println(">");

  }

  public void internalEntityDecl(String name,
   String value) throws SAXException {

    if (!name.startsWith("%")) { // ignore parameter entities
      System.out.println("<!ENTITY " + name + " \""
       + value + "\">");
    }

  }

  public void externalEntityDecl(String name,
   String publicID, String systemID) throws SAXException {

    if (!name.startsWith("%")) { // ignore parameter entities
      if (publicID != null) {
        System.out.println("<!ENTITY " + name + " PUBLIC \""
         + publicID + "\" \"" + systemID + "\">");

      }
      else {
        System.out.println("<!ENTITY " + name + " SYSTEM \""
         + systemID + "\">");
      }
    }

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java DTDMerger URL");
      return;
    }
    String document = args[0];

    XMLReader parser = null;
    try {
      parser = XMLReaderFactory.createXMLReader();
      DeclHandler handler = new DTDMerger();
      parser.setProperty(
       "http://xml.org/sax/properties/declaration-handler",
       handler);
      parser.parse(document);
    }
    catch (SAXNotRecognizedException e) {
      System.err.println(parser.getClass()
       + " does not support declaration handlers.");
    }
    catch (SAXNotSupportedException e) {
      System.err.println(parser.getClass()
       + " does not support declaration handlers.");

    }
    catch (SAXException e) {
      System.err.println(e);
      // As long as we finished with the DTD we really don't care
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }

  }

}

I ran this program across the start of an XHTML document (actually the XHTML 1.1 specification itself, though that detail doesn’t really matter since it’s the DTD we care about here, not the instance document. In fact, the instance document doesn’t even need to be well-formed as long as the error isn’t spotted until after the DOCTYPE declaration has been read.) Here’s the beginning of the merged DTD:

%java DTDMerger http://www.w3.org/TR/xhtml11
<!ATTLIST a onfocus CDATA #IMPLIED >
<!ATTLIST a onblur CDATA #IMPLIED >
<!ATTLIST form onsubmit CDATA #IMPLIED >
<!ATTLIST form onreset CDATA #IMPLIED >
<!ATTLIST label onfocus CDATA #IMPLIED >
…

If what you really want to know is the content specification for a particular element type, the output from this program is a lot easier to read than the original DTD. For example, here’s the original ELEMENT declaration for the p element:

<!ENTITY % p.element  "INCLUDE" >
<![%p.element;[
<!ENTITY % p.content
     "( #PCDATA | %Inline.mix; )*" >
<!ENTITY % p.qname  "p" >
<!ELEMENT %p.qname;  %p.content; >
<!-- end of p.element -->]]>

Now here’s the merged version:

<!ELEMENT p
  (#PCDATA|br|span|em|strong|dfn|code|samp|kbd|var|cite|abbr
   |acronym|q|tt|i|b|big|small|sub|sup|bdo|a|img|map|object|input
   |select|textarea|label|button|ruby|ins|del|script|noscript)* >

I think you’ll agree the second version is a lot easier to follow and understand. There are good and valid reasons to write the original DTD in the form used by the first declaration. However, that’s just not a form you want to present to a human being instead of a computer.

Xerces Custom Features

Individual parsers generally have a set of their own custom features and properties that control their own special capabilities. This allows you to configure a parser without having to go outside the standard SAX API, and thus binding your code to one specific parser. There’s generally no problem with using these non-standard features. Just make sure you watch out for SAXNotRecognizedException in case you later need to switch to a different parser that doesn’t support the same features.

For purposes of illustration, I’ll look at the custom features in Xerces 1.4.3, all of which are in the http://apache.org/xml/features/ hierarchy. Other parsers will have some features similar to these and some unique ones of their own. However, all will be in a domain of that parser vendor.

http://apache.org/xml/features/validation/schema

If true, Xerces will use any XML schemas it finds for applying default attribute values, assigning types to attributes, and possibly validation. (Validation also depends on the http://xml.org/sax/features/validation feature.) If false, then Xerces won’t use schemas at all, just the DTD. The default is true, use the schema if present.

http://apache.org/xml/features/validation/schema-full-checking

A number of features of the W3C XML Schema Language are extremely compute intensive. For example, the rather technical requirement for “Unique Particle Attribution” requires that given any element it’s possible to tell which part of a schema that element matches without considering the items the element contains or the elements that follow that element. However, this is extremely difficult to state, much less implement, in a precisely correct way. Consequently, Xerces by default skips these expensive checks. However, if you want them performed despite their cost you can turn them on by setting this feature to true.

http://apache.org/xml/features/validation/dynamic

If true, then Xerces will only attempt to validate documents that have a DOCTYPE declaration or an xsi:schemaLocation attribute. It will not attempt to validate merely well-formed documents that have neither.

http://apache.org/xml/features/validation/warn-on-duplicate-attdef

It is technically legal to declare the same attribute twice, and the declarations don’t even have to be compatible. For example,

<!ATTLIST Order id ID #IMPLIED>
<!ATTLIST Order id CDATA #REQUIRED>

The parser simply picks the first declaration and ignores the rest. Nonetheless, this probably indicates a mistake in the DTD. If the warn-on-duplicate-attdef feature is true, then Xerces should warn of duplicate attribute declarations by invoking the warning() method in the registered ErrorHandler. The default is to warn of this problem.

http://apache.org/xml/features/validation/warn-on-undeclared-elemdef

It is technically legal to declare an attribute for an element that has not been declared. This might happen if you delete an ELEMENT declaration but forget to delete one of the ATTLIST declarations for that element. Nonetheless, this almost certainly indicates a mistake. If this feature is true, then Xerces will warn of attribute declarations for non-existent elements. The default is to warn of this problem.

http://apache.org/xml/features/allow-java-encodings

By default Xerces only recognizes the standard encoding names like ISO-8859-1 and UTF-8. However, if this feature is turned on, then Xerces will also recognize Java style encoding names like 8859_1 and UTF8. The default is false.

http://apache.org/xml/features/continue-after-fatal-error

If true, Xerces will continue to parse a document after it detects a well-formedness error in order to detect and report more errors. This is useful for debugging because it allows you to be informed of and correct multiple errors before parsing a document again. The default is false. Note that the only thing Xerces will do after it sees the first well-formedness error, is look for more errors. It will not invoke any methods in any of the callback interfaces except ErrorHandler.

http://apache.org/xml/features/nonvalidating/load-dtd-grammar

If true, Xerces will attach default attributes to elements and specify attribute types even if it isn’t validating. If false it won’t. The default is true; and if validation is turned on, this feature is automatically turned on and cannot be turned off.

http://apache.org/xml/features/nonvalidating/load-external-dtd

If true, Xerces will load the external DTD subset. If false it won’t. The default is true. If validation is turned on, this feature is automatically turned on and cannot be turned off.

Example 7.15 is a variation of the earlier SAXValidator program that uses Xerces custom features to provide as many warnings and errors as possible. It uses dynamic validation so it only reports validity errors if the document is in fact trying to be valid. It turns on all optional warnings. And it continues parsing after a fatal error so it can find and report any more errors it spots in the document. This program is more useful for checking documents than the earlier generic program in Example 7.9. The downside is that it is totally dependent on the Xerces parser. It will not run with any other parser. Indeed it might even have troubles with earlier or later versions of Xerces. (I wrote this with 1.4.3.)

Example 7.15. Making maximal use of Xerces’s special abilities

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.IOException;

public class XercesChecker implements ErrorHandler {

  // Flag to check whether any errors have been spotted.
  private boolean valid = true;

  public boolean isValid() {
    return valid;
  }

  // If this handler is used to parse more than one document,
  // its initial state needs to be reset between parses.
  public void reset() {
    // Assume document is valid until proven otherwise
    valid = true;
  }

  public void warning(SAXParseException exception) {

    System.out.println("Warning: " + exception.getMessage());
    System.out.println(" at line " + exception.getLineNumber()
     + ", column " + exception.getColumnNumber());
    System.out.println(" in entity " + exception.getSystemId());

  }

  public void error(SAXParseException exception) {

    System.out.println("Error: " + exception.getMessage());
    System.out.println(" at line " + exception.getLineNumber()
     + ", column " + exception.getColumnNumber());
    // Unfortunately there's no good way to distinguish between
    // validity errors and other kinds of non-fatal errors
    valid = false;

  }

  public void fatalError(SAXParseException exception) {

    System.out.println("Fatal Error: " + exception.getMessage());
    System.out.println(" at line " + exception.getLineNumber()
     + ", column " + exception.getColumnNumber());
    System.out.println(" in entity " + exception.getSystemId());

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java XercesChecker URL");
      return;
    }
    String document = args[0];

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader(
       "org.apache.xerces.parsers.SAXParser"
      );
      XercesChecker handler = new XercesChecker();
      parser.setErrorHandler(handler);

      // This is a hack to fit some long lines of code that
      // follow between the margins of this printed page
      String features = "http://apache.org/xml/features/";

      // Turn on Xerces specific features
      parser.setFeature(features + "validation/dynamic", true);
      parser.setFeature(features
       + "validation/schema-full-checking", true);
      parser.setFeature(features
       + "validation/warn-on-duplicate-attdef", true);
      parser.setFeature(features
       + "validation/warn-on-undeclared-elemdef", true);
      parser.setFeature(features + "continue-after-fatal-error",
       true);
      parser.parse(document);
      if (handler.isValid()) {
        System.out.println(document + " is valid.");
      }
      else {
        // If the document isn't well-formed, an exception has
        // already been thrown and this has been skipped.
        System.out.println(document + " is well-formed.");
      }
    }
    catch (SAXParseException e) {
      System.out.print(document + " is not well-formed at ");
      System.out.println("Line " + e.getLineNumber()
       + ", column " +  e.getColumnNumber()
       + " in file " + e.getSystemId());
    }
    catch (SAXException e) {
      System.out.println("Could not check document because "
       + e.getMessage());
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }

  }

}

Here’s the beginning of the output from running it across one of my web pages that was supposed to be well-formed HTML, but proved not to be:

%java XercesChecker http://www.cafeconleche.org/
Fatal Error: The element type "br" must be terminated by the
 matching end-tag "</br>".
 at line 73, column 4
Fatal Error: The element type "br" must be terminated by the
 matching end-tag "</br>".
 at line 74, column 16
Fatal Error: The element type "dd" must be terminated by the
 matching end-tag "</dd>".
 at line 123, column 4
Fatal Error: The element type "br" must be terminated by the
 matching end-tag "</br>".
 at line 162, column 4
Fatal Error: The reference to entity "section" must end with
 the ';' delimiter.
 at line 183, column 78
 …

There were actually quite a few more errors that I’ve omitted here. The advantage of using XercesChecker instead of one of the earlier generic checking programs is that XercesChecker gives me a reasonably complete list of all the errors in one pass. I couldn’t necessarily do this with any off-the-shelf parser. With the earlier programs that stopped at the first fatal error, I’d have to fix one error, retest, fix the next error, retest, and so on until I had fixed the final error.

Xerces Custom Properties

DTDs require instance documents to specify what DTDs they should be validated against. While often useful, this can be dangerous. For example, imagine that you’ve written an order processing system that accepts XML documents containing orders from many heterogenous systems around the world. You can’t trust the people sending you orders to necessarily send them in the correct format, so as a first step you validate every order received. If the order document is invalid, your system rejects it.

However, this system has a flaw. Since the documents themselves specify which DTD they’ll be validated against, hackers can introduce bad data into your system by replacing the system identifier for your DTD with a URI for a DTD on a site they control. Then they can send you a document that will test as valid, even though it’s not, because it’s being validated against the wrong DTD!

For this and other reasons, the schema specification explicitly states that the xsi:schemaLocation and xsi:noNamespaceSchemaLocation attributes are not the only way to attach a schema to an instance document. The client application parsing a document is allowed to override the schema locations given in the document with schemas of its own choosing. For this purpose, Xerces has two custom properties:

  • http://apache.org/xml/properties/schema/external-schemaLocation

  • http://apache.org/xml/properties/schema/external-noNamespaceSchemaLocation

Both of these properties are strings telling the parser where a schema for elements in particular namespaces (or no namespace) can be found. They have the same syntax as the xsi:schemaLocation and xsi:noNamespaceSchemaLocation attributes in instance documents. For instance, this code fragment says that elements not in any namespace should be validated against the schema found at the relative URL orders.xsd:

parser.setProperty(
 "http://apache.org/xml/properties/schema/"
 + "external-noNamespaceSchemaLocation", "orders.xsd");

This code fragment says that elements in the http://schemas.xmlsoap.org/soap/envelope/namespace should be validated against the schema found at the URL http://www.w3.org/2002/06/soap-envelope and that elements in the http://schemas.xmlsoap.org/soap/encoding/ namespace should be validated against the schema found at the URL http://www.w3.org/2002/06/soap-encoding:

parser.setProperty(
 "http://apache.org/xml/properties/schema/external-SchemaLocation",
 "http://schemas.xmlsoap.org/soap/envelope/ "
 + "http://www.w3.org/2002/06/soap-envelope "
 + "http://schemas.xmlsoap.org/soap/encoding/ "
 + "http://www.w3.org/2002/06/soap-encoding");

If these properties are used and xsi:schemaLocation and/or xsi:noNamespaceSchemaLocation attributes are present in the instance document, then the schemas named by the properties take precedence.

These properties are only available in Xerces. Other parsers may support something similar, but if so they’ll place it at their own URL. In fact, as I write this Sun has just proposed adding http://java.sun.com/xml/jaxp/properties/schemaLanguage and http://java.sun.com/xml/jaxp/properties/schemaLocation properties to JAXP. The rough idea is the same, though Sun’s proposal would allow supporting arbitrary schema languages and allow the schemaLocation property to have a value from which the schema itself could be read rather than merely giving the location of the schema. For instance, it could be an InputStream or an InputSource object. Other parsers will doubtless implement this in other ways.

Prev | Up | Next
Exceptions and Errors | Home | DTDHandler
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified November 10, 2001


DTDHandler
Prev Chapter 7. The XMLReader Interface Next

DTDHandler

SAX is mostly about the instance document, not the DTD or schema. However, given a validating parser, or at least an internal DTD subset, the DTD can affect the contents of the instance document in six ways:

  1. It can provide default values for attributes.

  2. It can assign types to attributes which affects their normalized value.

  3. It can distinguish between ignorable and non-ignorable white space.

  4. It can declare general entities.

  5. It can declare unparsed entities.

  6. It can declare notations.

The first four are resolved silently. For instance, when applying a default value for an attribute to an element, the parser simply adds that attribute to the Attributes object it passes to startElement(). It doesn’t tell you that it’s done it. It just does it.

The DTDHandler interface covers the last two. Since notations and unparsed entities are so infrequently used, they’re not made a part of the main ContentHandler interface. Instead they’re given their own callback interface that’s just for working with notations and unparsed entities, DTDHandler. This is summarized in Example 7.16 Those few programmers who need this functionality can use it. Everyone else can ignore it.

Example 7.16. The DTDHandler interface

package org.xml.sax;

public interface DTDHandler {

  public void notationDecl(String name, String publicID,
   String systemID) throws SAXException;

  public void unparsedEntityDecl(String name, String publicID,
   String systemID, String notationName) throws SAXException;

}

Like other callback interfaces, programmers implement this interface in a class of their own choosing. That concrete instantiation is registered with the XMLReader through its setDTDHandler() method. For parallelism, there’s also a getDTDHandler() method though it isn’t much needed in practice:

public void setDTDHandler(DTDHandler handler);
public DTDHandler getDTDHandler();

As with the other callback interfaces, you can uninstall a DTDHandler by passing null to setDTDHandler().

The most common thing to do with a DTDHandler is simply store all the information provided about the notations and unparsed entities. Then the ContentHandler can refer back to this when it needs to resolve an unparsed entity. For instance, Example 7.17 is a simple DTDHandler implementation that stores the notations and unparsed entities declared in the DTD in two hash tables.

Example 7.17. A caching DTDHandler

import org.xml.sax.*;
import java.util.Hashtable;

public class UnparsedCache implements DTDHandler {

  private Hashtable notations = new Hashtable();
  private Hashtable entities = new Hashtable();

  public void notationDecl(String name, String publicID,
   String systemID) {

    System.out.println(name);
    notations.put(name, new Notation(name, publicID, systemID));

  }

  public void unparsedEntityDecl(String name, String publicID,
   String systemID, String notationName) {

    entities.put(name, new UnparsedEntity(name, publicID,
     systemID, notationName));

  }

  public UnparsedEntity getUnparsedEntity(String name) {
    System.out.println("Getting " + name);
    return (UnparsedEntity) entities.get(name);
  }

  public Notation getNotation(String name) {
    System.out.println("Getting " + name);
    return (Notation) notations.get(name);
  }

}

For the convenience of tracking the several strings associated with each notation and unparsed entity, I wrap each one in a very simple class that just has a constructor, some getter methods, the equals() and hashCode() methods needed to store these objects in hash tables, and a toString() method for convenient output. The Notation class is shown in Example 7.18. The UnparsedEntity class is shown in Example 7.19. Once you learn about DOM, an alternative would be to use that API’s Notation and Entity classes instead.

Example 7.18. A Notation utility class

public class Notation {

  private String name;
  private String publicID;
  private String systemID;

  public Notation(String name, String publicID,
   String systemID) {

    this.name = name;
    this.publicID = publicID;
    this.systemID = systemID;

  }

  public String getName() {
    return this.name;
  }

  public String getSystemID() {
    return this.systemID;
  }

  public String getPublicID() {
    return this.publicID;
  }

  public boolean equals(Object o) {

    if (o instanceof Notation) {
      Notation n = (Notation) o;
      // Well-formedness requires every notation to have
      // at least a SYSTEM or a PUBLIC ID so both should not be
      // simultaneously null as long as the UnparsedCache built
      // this object
      if (publicID == null) {
        return name.equals(n.name)
         && systemID.equals(n.systemID);
      }
      else if (systemID == null) {
        return name.equals(n.name)
         && publicID.equals(n.publicID);
      }
      else {
        return name.equals(n.name)
         && publicID.equals(n.publicID)
         && systemID.equals(n.systemID);
      }
    }
    return false;

  }

  public int hashCode() {

    if (publicID == null) {
      return name.hashCode() ^ systemID.hashCode();
    }
    else if (systemID == null) {
      return name.hashCode() ^ publicID.hashCode();
    }
    else {
      return name.hashCode() ^ publicID.hashCode()
       ^ systemID.hashCode();
    }

  }

  public String toString() {

    StringBuffer result = new StringBuffer(name);
    if (publicID != null) {
      result.append(" PUBLIC ");
      result.append(publicID);
      if (systemID != null) {
        result.append(" ");
        result.append(systemID);
      }
    }
    else {
      result.append(" SYSTEM ");
      result.append(systemID);
    }
    return result.toString();

  }

}

Example 7.19. An UnparsedEntity utility class

public class UnparsedEntity {

  private String name;
  private String publicID;
  private String systemID;
  private String notationName;

  public UnparsedEntity(String name, String publicID,
   String systemID, String notationName) {

    this.name = name;
    this.publicID = publicID;
    this.systemID = systemID;
    this.notationName = notationName;

  }

  public String getName() {
    return this.name;
  }

  public String getSystemID() {
    return this.systemID;
  }

  public String getPublicID() {
    return this.publicID;
  }

  public String getNotationName() {
    return this.notationName;
  }

  public boolean equals(Object o) {

    if (o instanceof UnparsedEntity) {
      UnparsedEntity entity = (UnparsedEntity) o;
      if (publicID == null) {
        return name.equals(entity.name)
         && systemID.equals(entity.systemID)
         && notationName.equals(entity.notationName);
      }
      else {
        return name.equals(entity.name)
         && systemID.equals(entity.systemID)
         && publicID.equals(entity.publicID)
         && notationName.equals(entity.notationName);
      }
    }
    return false;

  }

  public int hashCode() {

    if (publicID == null) {
      return name.hashCode() ^ systemID.hashCode()
       ^ notationName.hashCode();
    }
    else {
      return name.hashCode() ^ publicID.hashCode()
       ^ systemID.hashCode() ^ notationName.hashCode();
    }

  }

  public String toString() {

    StringBuffer result = new StringBuffer(name);
    if (publicID == null) {
      result.append(" PUBLIC ");
      result.append(publicID);
    }
    else {
      result.append(" SYSTEM ");
    }
    result.append(" ");
    result.append(systemID);

    return result.toString();
  }

}

When you later encounter an attribute of type ENTITY, ENTITIES, or NOTATION in the ContentHandler, you can use the getEntity() and getNotation() methods to return the relevant data for that item. For example, Example 7.20 is a simple program to list the unparsed entities and notations discovered in an XML document.

Example 7.20. A program that lists the unparsed entities and notations used in an XML document

import org.xml.sax.*;
import org.xml.sax.helpers.*;
import java.io.IOException;
import java.util.StringTokenizer;

public class EntityLister extends DefaultHandler {

  private UnparsedCache cache;

  public EntityLister(UnparsedCache cache) {
    this.cache = cache;
  }

  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes attributes) {

    for (int i = 0; i < attributes.getLength(); i++) {

      if (attributes.getType(i).equals("NOTATION")) {
        Notation n = cache.getNotation(attributes.getValue(i));
        System.out.println("Element " + qualifiedName
         + " has notation " + n);
      }
      else if (attributes.getType(i).equals("ENTITY")) {
        UnparsedEntity e = cache.getUnparsedEntity(
         attributes.getValue(i));
        System.out.println("Entity: " + e);
      }
      else if (attributes.getType(i).equals("ENTITIES")) {
        String entityNames = attributes.getValue(i);
        StringTokenizer st
         = new StringTokenizer(entityNames);
        while (st.hasMoreTokens()) {
           String name = st.nextToken();
           UnparsedEntity e = cache.getUnparsedEntity(name);
           System.out.println("Entity: " + e);
        }

      }

    }

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java EntityLister URL");
      return;
    }
    String document = args[0];

    try {
      XMLReader parser = XMLReaderFactory.createXMLReader();

      // I want to use qualified names
      parser.setFeature(
       "http://xml.org/sax/features/namespace-prefixes", true);

      UnparsedCache cache = new UnparsedCache();
      parser.setDTDHandler(cache);
      parser.setContentHandler(new EntityLister(cache));

      parser.parse(document);
    }
    catch (Exception e) {
      System.out.println("Could not read document because "
       + e.getMessage());
    }

  }

}

It took me a while to find an XML document in the wild that actually used notations and unparsed entities. However, David Carlisle pointed out to me that DocBook uses notations to identify preformatted elements in which white space should be preserved, and since this book is written in Docbook, I ran EntityLister across a rough draft of this chapter. Here’s what came out:

% java EntityLister xmlreader.xml
Element screen has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
Element screen has notation linespecific SYSTEM linespecific
Element programlisting has notation linespecific SYSTEM linespecific
…
Prev | Up | Next
Features and Properties | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 26, 2002


Summary
Prev Chapter 7. The XMLReader Interface Next

Summary

The SAX XMLReader interface represents an XML parser. The actual implementation class will vary from one parser to the next, but generally you’ll only interact with it through the parser-independent methods in XMLReader. Parser-specific instances of this interface are built by the XMLReaderFactory factory class.

The XMLReader’s parse() method actually reads the XML document. One parser object can read multiple documents in series. However, parsers are not thread safe or reentrant and should not be used to read documents in parallel. The source from which the XML document is read, be it an InputStream, Reader, or URI, is wrapped inside an org.xml.sax.InputSource object. An EntityResolver can be used to replace one InputSource with a different InputSource behind the parser’s back. If anything goes wrong during the parse process, this method throws a SAXException or an IOException

The parser is configured by setting features and properties. Both features and properties have names given by absolute URIs. Features have boolean values, true or false. Properties have object values. Standard features have names beginning with http://xml.org/sax/features/ and include http://xml.org/sax/features/external-general-entities, http://xml.org/sax/features/external-parameter-entities, http://xml.org/sax/features/string-interning, and http://xml.org/sax/features/validation. Standard properties have names beginning with http://xml.org/sax/properties/ and include http://xml.org/sax/properties/declaration-handler, http://xml.org/sax/properties/dom-node, http://xml.org/sax/properties/lexical-handler, and http://xml.org/sax/properties/xml-string.

This is not an exhaustive list. Parser vendors add custom features and properties with names in their own domain such as http://apache.org/xml/features/validation/schema and http://apache.org/xml/properties/schema/external-schemaLocation to configure functionality unique to their parser. Of course, not all features and properties, not even all standard features and properties, are supported by any one parser. If you try to set a feature or property that the parser is unfamiliar with and never allows under any circumstances, setFeature()/setProperty() throws a SAXNotRecognizedException. If you try to set a feature or property that the parser recognizes but cannot set at the current time, setFeature()/setProperty() throws a SAXNotSupportedException.

The parser reports the content of the XML document to methods in one or more callback interfaces. The most basic such callback interface and the one that accounts for 90% of what you need is ContentHandler. Other callback interfaces for special needs include ErrorHandler for reporting fatal errors, errors, and warnings and DTDHandler for keeping track of unparsed entity and notation declarations. In addition there are two optional callback interfaces registered by setting properties, LexicalHandler for reporting comments, parsed entities, and CDATA sections; and DeclHandler for reporting ELEMENT, ATTLIST, and ENTITY declarations.

Prev | Up | Next
DTDHandler | Home | Chapter 8. SAX Filters
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 09, 2002


Chapter 8. SAX Filters
Prev Next

Chapter 8. SAX Filters

Table of Contents

The Filter Architecture
The XMLFilter interface
Content Filters
Filtering Tags
Filtering Elements
Filtering attributes
Filters that add content
Filters vs. Transforms
The XMLFilterImpl Class
Parsing non-XML Documents
Multihandler adapters
Summary

In all of XML, I have found nothing quite so hard to understand yet easy to do as writing SAX filters. For a long time, it felt like I had a mental block preventing me from grokking just how filters worked, and yet every time I wrote one it almost always worked on the first try. In fact, even when I was convinced that the code I had written could not possibly work, it did. I can’t decide whether this is an example of wonderful or awful API design.

The basic idea of filters is that an XMLReader, instead of receiving XML text directly from a file, socket, or other source, receives already parsed events from another XMLReader. It can change these events before passing them along to the client application through the usual methods of ContentHandler and the other callback interfaces. For example, it can add a unique ID attribute to every element or delete all elements in the SVG namespace from the input stream.

The Filter Architecture

Figure 8.1 diagrams the normal course of XML processing. A client application instructs a parser, represented in SAX by an XMLReader object, to read the text of an XML document. As it reads, the parser calls back to the client application’s ContentHandler.

Figure 8.1. The XML parsing process

Figure 8.2 diagrams the course of XML processing with a filter. A client application instructs the filter, represented in SAX by an XMLFilter object, to read the text of an XML document. The filter then instructs the parser to read the text of an XML document. As it reads, the parser calls back to the filter’s ContentHandler. The filter’s ContentHandler then calls back to the client application’s ContentHandler.

Figure 8.2. XML parsing with a filter

Since the filter sits in the middle between the real parser and the client application, it can change the stream of events that gets passed back and forth between the two. For example, it can convert VML to SVG on the fly. It can replace xinclude:include elements with the documents they point to. It can add namespaces to elements and attributes that don’t normally have them. Alternately it can work with the stream without actually changing the data itself. For example, it could log all SOAP requests that pass through it to a database. This wouldn’t necessarily change the data passing through the filter in any way, but it could help implement transactions with rollback and/or journaling to protect against data corruption in the event of a system crash.

The XMLFilter interface is a subinterface of XMLReader. Therefore, rather than chaining a filter directly to the parser that’s reading the actual document, you can instead chain it to another filter. And this filter can be chained to another filter, and this filter can be chained to another filter, and so on, for as many filters as necessary, as diagrammed in Figure 8.3. For example, this allows you to add namespaces to a document, convert VML to SVG, and resolve XIncludes, all while parsing a single document. The document the client application receives can actually be quite different from the text file stored on a disk somewhere, depending on how many filters there are and how much they change the original data.

Figure 8.3. XML parsing with multiple filters

As a final trick, filters can present data that is not XML to an application as if it were XML. In other words, you can write a parser for some non-XML data such as tab-delimited text files, and provide an XMLReader interface through a filter to hide the fact that what’s being parsed isn’t really XML!

Prev | Up | Next
Summary | Home | The XMLFilter interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 09, 2002


The XMLFilter interface
Prev Chapter 8. SAX Filters Next

The XMLFilter interface

Example 8.1 shows the actual code for the XMLFilter interface. Besides the methods it inherits from the XMLReader superinterface, XMLFilter has just two new methods, getParent() and setParent(). The parent of a filter is the XMLReader to which the filter delegates most of its work. (In the context of SAX filters, the parent is not normally understood to be the superclass of the filter class.)

Example 8.1. The XMLFilter interface

package org.xml.sax;

public interface XMLFilter extends XMLReader {

  public void      setParent(XMLReader parent);
  public XMLReader getParent();

}

A class that implements this interface must provide a minimum of 16 methods, the getParent() and setParent() methods declared here and the 14 methods of the XMLReader superinterface. Example 8.2 is a minimal XML filter that implements all of these methods but doesn’t actually do anything.

Example 8.2. A filter that blocks all events

import org.xml.sax.*;

public class OpaqueFilter implements XMLFilter {

  private XMLReader parent;

  public void setParent(XMLReader parent) {
    this.parent = parent;
  }

  public XMLReader getParent() {
    return this.parent;
  }

  public boolean getFeature(String name)
   throws SAXNotRecognizedException {
    throw new SAXNotRecognizedException(name);
  }

  public void setFeature(String name, boolean value)
   throws SAXNotRecognizedException {
    throw new SAXNotRecognizedException(name);
  }

  public Object getProperty(String name)
   throws SAXNotRecognizedException {
    throw new SAXNotRecognizedException(name);
  }

  public void setProperty(String name, Object value)
   throws SAXNotRecognizedException {
    throw new SAXNotRecognizedException(name);
  }

  public void setEntityResolver(EntityResolver resolver) {}
  public EntityResolver getEntityResolver() {
    return null;
  }

  public void setDTDHandler(DTDHandler handler) {}
  public DTDHandler getDTDHandler() {
    return null;
  }

  public void setContentHandler(ContentHandler handler) {}
  public ContentHandler getContentHandler() {
    return null;
  }

  public void setErrorHandler(ErrorHandler handler) {}
  public ErrorHandler getErrorHandler() {
    return null;
  }

  public void parse(InputSource input) {}
  public void parse(String systemID) {}

}

The effect of attaching this filter to a parser is to totally block events. It’s like a brick wall between the application and the data in the XML document. A client application that wants to use this filter (though I really can’t imagine why one would) would construct an instance of it and an instance of a real parser, then pass the real parser to the filter’s setParent() method.

  XMLReader parser = XMLReaderFactory.createXMLReader();
  OpaqueFilter filter = new OpaqueFilter();
  filter.setParent(parser);

From this point forward, the client application should only interact with the filter. It should forget that the original parser exists. Going behind the back of the filter, for instance, by calling setContentHandler() on parser instead of on filter, runs the risk of confusing the filter by violating constraints it expects to be true. In fact, if at all possible, you should eliminate any references to the original parser so that you can’t accidentally access it later. For example,

  XMLReader parser = XMLReaderFactory.createXMLReader();
  OpaqueFilter filter = new OpaqueFilter();
  filter.setParent(parser);
  parser=filter;

In some cases the filter may set up its own parent parser, typically in its constructor. This avoids the need for the client application to provide an XMLReader to the filter. For example,

  public OpaqueFilter(XMLReader parent) {
    this.parent = parent;
  }

  public OpaqueFilter() throws SAXException {
    this(XMLReaderFactory.createXMLReader());
  }

You might even design the setParent() method so it’s impossible to change the parent parser once it’s initially been set in the constructor. For example,

  public void setParent(XMLReader parent) {
    throw new UnsupportedOperationException(
     "Can’t change this filter’s parent"
    );
  }

However, this does tend to limit the flexibility of the filter. In particular, it prevents you from putting it in the middle of a long chain of filters.

Example 8.3 is a marginally more interesting implementation of the XMLFilter interface. It delegates to the parent XMLReader by forwarding all method calls from the client application. It does not change or filter anything.

Example 8.3. A filter that filters nothing

import org.xml.sax.*;
import java.io.IOException;

public class TransparentFilter implements XMLFilter {

  private XMLReader parent;

  public void setParent(XMLReader parent) {
    this.parent = parent;
  }

  public XMLReader getParent() {
    return this.parent;
  }

  public boolean getFeature(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    return parent.getFeature(name);
  }

  public void setFeature(String name, boolean value)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    parent.setFeature(name, value);
  }

  public Object getProperty(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    return parent.getProperty(name);
  }

  public void setProperty(String name, Object value)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    parent.setProperty(name, value);
  }

  public void setEntityResolver(EntityResolver resolver) {
    parent.setEntityResolver(resolver);
  }

  public EntityResolver getEntityResolver() {
    return parent.getEntityResolver();
  }

  public void setDTDHandler(DTDHandler handler) {
    parent.setDTDHandler(handler);
  }

  public DTDHandler getDTDHandler() {
    return parent.getDTDHandler();
  }

  public void setContentHandler(ContentHandler handler) {
    parent.setContentHandler(handler);
  }

  public ContentHandler getContentHandler() {
    return parent.getContentHandler();
  }

  public void setErrorHandler(ErrorHandler handler) {
    parent.setErrorHandler(handler);
  }

  public ErrorHandler getErrorHandler() {
    return parent.getErrorHandler();
  }

  public void parse(InputSource input)
   throws SAXException, IOException {
    parent.parse(input);
  }

  public void parse(String systemId)
   throws SAXException, IOException {
    parent.parse(systemId);
  }

}

Of course, in most cases you’re not going to go to either of these extremes. You’re going to pass some events through unchanged, block others, and modify still others. Let’s continue with a filter that adds a property to the list of those normally supported by an XML parser. This property will provide the wall-clock time needed to parse an XML document, and might be useful for benchmarking. I’ll write it as a filter so that it can be attached to different underlying parsers and used in benchmarks that include various content handlers.

The property name will be http://cafeconleche.org/properties/wallclock/. The value will be a java.lang.Long object containing the number of milliseconds needed to parse the last document. This can be stored in a private field initialized to null:

  private Long wallclock = null;

The wallclock time is available only after the parse() method has returned. At other times, requesting this property throws a SAXNotSupportedException. This will be a read-only property so trying to set it will always throw a SAXNotSupportedException. This will be implemented through the setProperty() and getProperty() methods:

  public Object getProperty(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException {

    if ("http://cafeconleche.org/properties/wallclock/"
     .equals(name)) {
      if (wallclock != null) {
        return wallclock;
      }
      else {
        throw
         new SAXNotSupportedException("Timing not available");
      }
    }
    return parent.getProperty(name);

  }

  public void setProperty(String name, Object value)
   throws SAXNotRecognizedException, SAXNotSupportedException {

    if ("http://cafeconleche.org/properties/wallclock/"
     .equals(name)) {
      throw new SAXNotSupportedException(
       "Wallclock property is read-only");
    }
    parent.setProperty(name, value);

  }

For any property other than http://cafeconleche.org/properties/wallclock/, these calls just delegate the work to the parent parser.

The parse() method is responsible for tracking the wallclock time. I’ll put the work in the parse() method that takes an InputSource as an argument and call this method from the other overloaded parse() method that takes a system ID as the argument.

Using the filter enables some standard benchmarking techniques. First, I’ll read the entire document into a byte array named cache so that it can be parsed from memory. This will eliminate most of the I/O time which would otherwise be likely to swamp the actual parsing time, especially if the test document were read from a slow network connection. This actually requires separate handling for the three possible sources an InputSource may offer: character stream, byte stream, and system ID:

    ByteArrayOutputStream out = new ByteArrayOutputStream();
    Reader charStream = input.getCharacterStream();
    InputStream byteStream = input.getByteStream();

    String encoding = null; // I will only set this variable if
                            // we have a reader because in this
                            // case we know the encoding is UTF-8
                            // regardless of what the encoding
                            // declaration says
    if (charStream != null) {
      OutputStreamWriter filter
       = new OutputStreamWriter(out, "UTF-8");
      int c;
      while ((c = charStream.read()) != -1) filter.write(c);
      encoding = "UTF-8";
    }
    else if (byteStream != null) {
      int c;
      while ((c = byteStream.read()) != -1) out.write(c);
    }
    else {
      URL u = new URL(input.getSystemId());
      InputStream in = u.openStream();
      int c;
      while ((c = in.read()) != -1) out.write(c);
    }
    out.flush();
    out.close();
    byte[] cache = out.toByteArray();

Next, I’ll warm up the JIT with ten untimed parses of the document before I begin taking measurements:

    for (int i=0; i < 10; i++) {
      InputStream in = new ByteArrayInputStream(cache);
      is.setByteStream(in);
      parent.parse(is);
    }

Finally I’ll parse the same document 1000 times and set wallclock to the average of the 1000 parses:

    Date start = new Date();
    for (int i=0; i < 1000; i++) {
      InputStream in = new ByteArrayInputStream(cache);
      is.setByteStream(in);
      parent.parse(is);
    }
    Date finish = new Date();
    long totalTime = finish.getTime() - start.getTime();

    // Average the time
    this.wallclock = new Long(totalTime/1000);

Example 8.4 shows the complete benchmarking filter. Besides the previously described methods, it contains implementations of the other XMLReader methods that all forward their arguments to the equivalent method in the parent parser.

Example 8.4. A filter that times all parsing

import org.xml.sax.*;
import java.io.*;
import java.util.Date;
import java.net.URL;

public class WallclockFilter implements XMLFilter {

  private XMLReader parent;
  private Long wallclock = null;

  public Object getProperty(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException {

    if ("http://cafeconleche.org/properties/wallclock/"
     .equals(name)) {
      if (wallclock != null) {
        return wallclock;
      }
      else {
        throw
         new SAXNotSupportedException("Timing not available");
      }
    }
    return parent.getProperty(name);

  }

  public void setProperty(String name, Object value)
   throws SAXNotRecognizedException, SAXNotSupportedException {

    if ("http://cafeconleche.org/properties/wallclock/"
     .equals(name)) {
      throw new SAXNotSupportedException(
       "Wallclock property is read-only");
    }
    parent.setProperty(name, value);

  }

  public void setParent(XMLReader parent) {
    this.parent = parent;
  }

  public XMLReader getParent() {
    return this.parent;
  }

  public void parse(InputSource input)
   throws SAXException, IOException {

    //Reset the time
    this.wallclock = null;

    // Cache the document
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    Reader charStream = input.getCharacterStream();
    InputStream byteStream = input.getByteStream();

    String encoding = null; // I will only set this variable if
                            // we have a reader because in this
                            // case we know the encoding is UTF-8
                            // regardless of what the encoding
                            // declaration says
    if (charStream != null) {
      OutputStreamWriter filter
       = new OutputStreamWriter(out, "UTF-8");
      int c;
      while ((c = charStream.read()) != -1) filter.write(c);
      encoding = "UTF-8";
    }
    else if (byteStream != null) {
      int c;
      while ((c = byteStream.read()) != -1) out.write(c);
    }
    else {
      URL u = new URL(input.getSystemId());
      InputStream in = u.openStream();
      int c;
      while ((c = in.read()) != -1) out.write(c);
    }
    out.flush();
    out.close();
    byte[] cache = out.toByteArray();

    InputSource is = new InputSource();
    if (encoding != null) is.setEncoding(encoding);

    // Warm up the JIT
    for (int i=0; i < 10; i++) {
      InputStream in = new ByteArrayInputStream(cache);
      is.setByteStream(in);
      parent.parse(is);
    }
    System.gc();

    // Parse 1000 times
    Date start = new Date();
    for (int i=0; i < 1000; i++) {
      InputStream in = new ByteArrayInputStream(cache);
      is.setByteStream(in);
      parent.parse(is);
    }
    Date finish = new Date();
    long totalTime = finish.getTime() - start.getTime();

    // Average the time
    this.wallclock = new Long(totalTime/1000);

  }

  public void parse(String systemID)
   throws SAXException, IOException {
    this.parse(new InputSource(systemID));
  }

  // Methods that delegate to the parent XMLReader
  public boolean getFeature(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    return parent.getFeature(name);
  }

  public void setFeature(String name, boolean value)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    parent.setFeature(name, value);
  }

  public void setEntityResolver(EntityResolver resolver) {
    parent.setEntityResolver(resolver);
  }

  public EntityResolver getEntityResolver() {
    return parent.getEntityResolver();
  }

  public void setDTDHandler(DTDHandler handler) {
    parent.setDTDHandler(handler);
  }

  public DTDHandler getDTDHandler() {
    return parent.getDTDHandler();
  }

  public void setContentHandler(ContentHandler handler) {
    parent.setContentHandler(handler);
  }

  public ContentHandler getContentHandler() {
    return parent.getContentHandler();
  }

  public void setErrorHandler(ErrorHandler handler) {
    parent.setErrorHandler(handler);
  }

  public ErrorHandler getErrorHandler() {
    return parent.getErrorHandler();
  }

}

We still need a driver class that constructs a filter XMLReader and a normal parser XMLReader, connects them to each other, and parses the test document. Example 8.5 is such a class that contains a simple main() method to benchmark a document named on the command line. No handlers are installed so it tests raw parsing time. If I wanted to test the behavior of different parsers with various callback interfaces, I could install them on the parser before parsing. After parsing, WallclockDriver reads the value of the wallclock property from the filter. The parser tested can be adjusted by setting different values for the org.xml.sax.driver system property.

Example 8.5. Parsing a document through a filter

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.IOException;

public class WallclockDriver {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java WallclockDriver URL");
      return;
    }
    String document = args[0];

    try {
      XMLFilter filter = new WallclockFilter();
      filter.setParent(XMLReaderFactory.createXMLReader());
      filter.parse(document);
      Long parseTime = (Long) filter.getProperty(
       "http://cafeconleche.org/properties/wallclock/");
       double seconds = parseTime.longValue()/1000.0;
      System.out.println("Parsing " + document + " took "
       + seconds + " seconds on average.");
    }
    catch (SAXException e) {
      e.printStackTrace();
      System.out.println(e);
    }
    catch (IOException e) {
        e.printStackTrace();
     System.out.println(
       "Due to an IOException, the parser could not check "
       + args[0]
      );
    }

  }

}

I ran the XML form of the second edition of the XML 1.0 specification through this program with a few different parsers using Sun’s Java Runtime Environment 1.3.1 on my 300 MHz Pentium II running Windows NT 4.0SP6. This isn’t a scientific test (At a minimum this would require testing many different documents on multiple virtual and physical machines, considering pauses that might be caused by garbage collection, making sure background processes were kept to a minimum, and taking multiple measurements to test reproducibility of the results), but the output’s nonetheless mildly interesting:

% java -Dorg.xml.sax.driver=gnu.xml.aelfred2.XmlReader
  WallclockDriver http://www.w3.org/TR/2000/REC-xml-20001006.xml
Parsing http://www.w3.org/TR/2000/REC-xml-20001006.xml
 took 1.209 seconds on average.
% java -Dorg.xml.sax.driver=org.apache.crimson.parser.XMLReaderImpl
  WallclockDriver http://www.w3.org/TR/2000/REC-xml-20001006.xml
Parsing http://www.w3.org/TR/2000/REC-xml-20001006.xml
 took 1.414 seconds on average.
% java -Dorg.xml.sax.driver=org.apache.xerces.parsers.SAXParser
  WallclockDriver http://www.w3.org/TR/2000/REC-xml-20001006.xml
Parsing http://www.w3.org/TR/2000/REC-xml-20001006.xml
 took 1.133 seconds on average.
% java -Dorg.xml.sax.driver=com.bluecast.xml.Piccolo
  WallclockDriver http://www.w3.org/TR/2000/REC-xml-20001006.xml
Parsing http://www.w3.org/TR/2000/REC-xml-20001006.xml
 took 0.849 seconds on average.

The four parsers I tested here were all fairly close to each other in raw performance. In fact, given that I’m testing the wallclock time instead of the actual time used by this program alone, I’d venture that the differences are all within the margin of error for the test. Of course, when choosing a parser, you’d want to run this across your own documents with your own content handlers in place.

Prev | Up | Next
Chapter 8. SAX Filters | Home | Content Filters
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


Content Filters
Prev Chapter 8. SAX Filters Next

Content Filters

By this point, you may have noticed a problem with the XMLFilter interface. It filters calls from the client application to the parser. However, most events are passed in the other direction from the parser to the client application through the various callback interfaces, especially ContentHandler. XMLFilter is set up exactly backwards for filtering these events. It filters calls from the client application to the parser, but not the much more important calls from the parser to the client application!

However, it’s possible to work around this. First intercept the handlers passed to methods like setContentHandler() and setDTDHandler(). Then replace them with handlers of your own so that your handlers receive the callback events from the parent parser. They can either pass them along to the client application’s handler methods or pass something different instead.

Filtering Tags

For example, let’s suppose you want to convert a RDDL document to pure XHTML. RDDL, the Resource Directory Description Language, is used for human and machine readable documents placed at the end of namespace URLs. RDDL is actually quite close to XHTML to begin with. It’s just an XHTML Basic document in which there’s one extra element, rddl:resource, which can appear anywhere a p element can appear, and can contain anything a div element can contain. The customary rddl prefix is mapped to the http://www.rddl.org/ namespace URL, and as usual the prefix can change as long as the URL remains the same. For example, this is a rddl:resource from the RDDL specification itself:

<rddl:resource id="rec-xhtml"
        xlink:title="W3C REC XHTML"
        xlink:role="http://www.w3.org/1999/xhtml"
        xlink:arcrole="http://www.rddl.org/purposes#reference"
        xlink:href="http://www.w3.org/tr/xhtml1"
        >
<li><a href="http://www.w3.org/tr/xhtml1">W3C XHTML 1.0</a></li>
</rddl:resource>

The filter needs to throw away the <rddl:resource> start-tag and </rddl:resource> end-tag while leaving everything else intact. Example 8.6 does this. The startElement() and endElement() methods pass everything except rddl:resource tags to the parent’s ContentHandler. However, rddl:resource tags are just dropped on the floor.

Example 8.6. A ContentHandler filter

import org.xml.sax.*;

public class RDDLStripper implements ContentHandler {

  public final static String RDDL_NAMESPACE
   = "http://www.rddl.org/";
  public final static String XHTML_NAMESPACE
   = "http://www.w3.org/1999/xhtml";

  private ContentHandler parent;

  public RDDLStripper(ContentHandler parent) {
    this.parent = parent;
  }

  // Filter out <rddl:resource> start-tags
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {

    if (localName.equals("resource")
        && namespaceURI.equals(RDDL_NAMESPACE)) {
      return; // having done nothing
    }
    else { // pass the element along
      parent.startElement(namespaceURI, localName, qualifiedName,
       atts);
    }

  }

  // Filter out </rddl:resource> end-tags
  public void endElement(String namespaceURI, String localName,
   String qualifiedName) throws SAXException {

    if (localName.equals("resource")
        && namespaceURI.equals(RDDL_NAMESPACE)) {
      return; // having done nothing
    }
    else {
      parent.endElement(namespaceURI, localName, qualifiedName);
    }

  }

  // Methods that pass data along unchanged:
  public void startDocument() throws SAXException {
    parent.startDocument();
  }

  public void startPrefixMapping(String prefix, String uri)
   throws SAXException {
    parent.startPrefixMapping(prefix, uri);
  }

  public void endPrefixMapping(String prefix)
   throws SAXException {
    parent.endPrefixMapping(prefix);
  }

  public void setDocumentLocator(Locator locator) {
    parent.setDocumentLocator(locator);
  }

  public void endDocument() throws SAXException {
    parent.endDocument();
  }

  public void characters(char[] text, int start, int length)
   throws SAXException {
    parent.characters(text, start, length);
  }

  public void ignorableWhitespace(char[] text, int start,
   int length) throws SAXException {
    parent.ignorableWhitespace(text, start, length);
  }

  public void processingInstruction(String target, String data)
   throws SAXException {
    parent.processingInstruction(target, data);
  }

  public void skippedEntity(String name)
   throws SAXException {
    parent.skippedEntity(name);
  }

}

The next step is to install a RDDLStripper as an XMLFilter’s ContentHandler. Example 8.7 does this. It’s just a slight variation on the earlier TransparentFilter. Here, however, the setContentHandler() method installs a RDDLStripper. [1]

Example 8.7. A filter that substitutes its own ContentHandler

import org.xml.sax.*;
import java.io.IOException;

public class RDDLFilter implements XMLFilter {

  private XMLReader parent;

  public void setParent(XMLReader parent) {
    this.parent = parent;
  }

  public XMLReader getParent() {
    return this.parent;
  }

  public void setContentHandler(ContentHandler handler) {
    parent.setContentHandler(new RDDLStripper(handler));
  }

  // Transparent pass-along methods
  public ContentHandler getContentHandler() {
    return parent.getContentHandler();
  }

  public boolean getFeature(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    return parent.getFeature(name);
  }

  public void setFeature(String name, boolean value)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    parent.setFeature(name, value);
  }

  public Object getProperty(String name)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    return parent.getProperty(name);
  }

  public void setProperty(String name, Object value)
   throws SAXNotRecognizedException, SAXNotSupportedException {
    parent.setProperty(name, value);
  }

  public void setEntityResolver(EntityResolver resolver) {
    parent.setEntityResolver(resolver);
  }

  public EntityResolver getEntityResolver() {
    return parent.getEntityResolver();
  }

  public void setDTDHandler(DTDHandler handler) {
    parent.setDTDHandler(handler);
  }

  public DTDHandler getDTDHandler() {
    return parent.getDTDHandler();
  }

  public void setErrorHandler(ErrorHandler handler) {
    parent.setErrorHandler(handler);
  }

  public ErrorHandler getErrorHandler() {
    return parent.getErrorHandler();
  }

  public void parse(InputSource input)
   throws SAXException, IOException {
    parent.parse(input);
  }

  public void parse(String systemId)
   throws SAXException, IOException {
    parent.parse(systemId);
  }

}

The final step is to write a client application which uses this filter to read XHTML and/or RDDL documents. Whichever kind you feed it, the client app should only see XHTML. For this client application, I’m going to use a ContentHandler that just prints the result on System.out. More specifically, I’m going to use David Megginson’s public domain com.megginson.sax.XMLWriter class. This class not only implements ContentHandler. It’s also an XMLFilter. However, since it uses a few features we haven’t discussed yet, detailed analysis of this class will have to wait a bit. In the meantime, all you really need to know is that XMLWriter logs all events the parser fires to a specified java.io.Writer in a well-formed way.

Example 8.8 demonstrates with a driver program that parses a document named by the first command line argument using the XMLFilter class named by the second command line argument. This test program will allow us to inspect the output of various filters in this chapter.

Example 8.8. A program that filters documents

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.*;
import com.megginson.sax.XMLWriter;

public class FilterTester {

  public static void main(String[] args) {

    if (args.length < 2) {
      System.out.println(
       "Usage: java FilterTester URL FilterClass");
      return;
    }
    String document = args[0];
    String filterClass = args[1];

    try {
      XMLFilter filter
       = (XMLFilter) Class.forName(filterClass).newInstance();
      filter.setParent(XMLReaderFactory.createXMLReader());
      filter.setContentHandler(
       new XMLWriter(new OutputStreamWriter(System.out))
      );
      filter.parse(document);
    }
    catch (SAXException e) {
      e.printStackTrace();
      System.out.println(e);
    }
    catch (IOException e) {
        e.printStackTrace();
     System.out.println(
       "Due to an IOException, the parser could not read "
       + args[0]
      );
    }
    catch (ClassCastException e) {
      System.out.println(filterClass
       + " does not implement org.xml.sax.XMLFilter");
    }
    catch (ClassNotFoundException e) {
      System.out.println(filterClass
       + " cannot be found in the CLASSPATH");
    }
    catch (InstantiationException e) {
      System.out.println(filterClass
       + " does not have a no-args constructor");
    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

For example, here’s the actual last few lines of the RDDL specification (modulo the usual white space adjustments):

<rddl:resource
        id="note-xlink2rdf"
        xlink:title="W3C NOTE XLink2RDF"
        xlink:role="http://www.w3.org/TR/html4/"
        xlink:arcrole="http://www.rddl.org/purposes#reference"
        xlink:href="http://www.w3.org/TR/xlink2rdf/"
        >
<li><a href="http://www.w3.org/TR/xlink2rdf/">W3C Note Harvesting
RDF Statements from XLinks</a></li>
</rddl:resource>
<rddl:resource
id="rec-modxhtml"
        xlink:title="W3C REC Modularization for XHTML"
        xlink:role="http://www.w3.org/1999/xhtml"
        xlink:arcrole="http://www.rddl.org/purposes#reference"
        xlink:href="http://www.w3.org/TR/xhtml-modularization/"
        >
<li><a href="http://www.w3.org/TR/xhtml-modularization/">W3C
Modularization of XHTML</a></li>
</rddl:resource>
</ol>
</div>
</body>
</html>

Now here’s some of the output when the RDDL specification was passed through this program. The rddl:resource tags have been removed:

% java -Dorg.xml.sax.driver=org.apache.xerces.parsers.SAXParser
  FilterTester http://www.rddl.org/ RDDLFilter
...
        <li><a href="http://www.w3.org/TR/xlink2rdf/">W3C Note
        Harvesting RDF Statements from XLinks</a></li>

        <li><a href="http://www.w3.org/TR/xhtml-modularization/">
        W3C Modularization of XHTML</a></li>

</ol>
</div>
</body>
</html>

The pattern followed here is pretty much exactly what was shown in Figure 8.2. If we revise that figure to include the specific class names used here, what you have is shown in Figure 8.4.

Figure 8.4. How data flows through the RDDL stripper program

Because SAX operates in a linear rather than hierarchical fashion, filtering the start-tag and end-tag of each rddl:resource element does not filter the contents of the rddl:resource element. In this case, that’s exactly what we want. RDDL is a little unusual in that it both contains and is contained in XHTML. Most applications that get mixed with XHTML such as SVG and MathML are contained in XHTML but do not themselves contain XHTML.

Filtering Elements

It’s easy enough to drop out any elements and that are not in the XHTML namespace. However, in the case of SVG, MathML and most other applications you’ll want to remove the content of these elements as well. I’ll assume that the namespace for text is the same as the namespace of the parent element. (This is not at all clear from the namespaces specification, but it makes sense in many cases.) To track the nearest namespace for non-elements, startElement() will push the element’s namespace onto a stack and endElement() will pop it off. Peeking at the top of the stack will tell you what namespace the nearest element uses. You can’t use NamespaceSupport() or startPrefixMapping() and endPrefixMapping() here because those track namespace declarations rather than the actual namespaces used. Example 8.9 is the ContentHandler that accomplishes this.

Example 8.9. A ContentHandler filter that throws away non-XHTML elements

import org.xml.sax.*;
import java.util.*;

public class PureXHTMLHandler implements ContentHandler {

  public final static String XHTML_NAMESPACE
   = "http://www.w3.org/1999/xhtml";

  private ContentHandler parent;
  private Stack namespaces;  // initialized in startDocument()

  public PureXHTMLHandler(ContentHandler parent) {
    this.parent = parent;
  }

  // Filter out anything that's not in the XHTML namespace
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {

    namespaces.push(namespaceURI);
    if (namespaceURI.equals(XHTML_NAMESPACE)) {
      parent.startElement(namespaceURI, localName, qualifiedName,
       atts);
    }
    // else do nothing

  }

  // Filter out anything that's not in the XHTML namespace
  public void endElement(String namespaceURI, String localName,
   String qualifiedName) throws SAXException {

    namespaces.pop();
    if (namespaceURI.equals(XHTML_NAMESPACE)) {
      parent.endElement(namespaceURI, localName, qualifiedName);
    }
    // else do-nothing

  }

  // simple utility method to determine whether or not the parent
  // element is an XHTML element
  private boolean inXHTML() {

    try {
      String namespace = (String) namespaces.peek();
      if (namespace.equals(XHTML_NAMESPACE)) return true;
      return false;
    }
    catch (EmptyStackException e) {
      // This means we're outside the root element in a
      // processing instruction. Such processing instructions are
      // legal in XHTML so I keep it.
      return true;
    }

  }

  public void characters(char[] text, int start, int length)
   throws SAXException {
    if (inXHTML()) parent.characters(text, start, length);
  }

  public void ignorableWhitespace(char[] text, int start,
   int length) throws SAXException {
    if (inXHTML()) parent.ignorableWhitespace(text, start, length);
  }

  public void processingInstruction(String target, String data)
   throws SAXException {
    if (inXHTML()) parent.processingInstruction(target, data);
  }

  public void skippedEntity(String name)
   throws SAXException {
    if (inXHTML()) parent.skippedEntity(name);
  }

  // I could track namespace declarations with these next two
  // methods and only pass along those for XHTML. However,
  // that is quite tricky because the endPrefixMapping()
  // method only reports the prefix, not the URI.
  public void startPrefixMapping(String prefix, String uri)
   throws SAXException {
    parent.startPrefixMapping(prefix, uri);
  }

  public void endPrefixMapping(String prefix)
   throws SAXException {
    parent.endPrefixMapping(prefix);
  }

  // Methods that pass data along unchanged:
  public void startDocument() throws SAXException {
    namespaces = new Stack();
    parent.startDocument();
  }

  public void endDocument() throws SAXException {
    parent.endDocument();
  }

  public void setDocumentLocator(Locator locator) {
    parent.setDocumentLocator(locator);
  }

}

One thing to consider: what happens when a non-XHTML document such as a pure SVG or MathML document is passed through this filter? In other words, what if the root element is not an XHTML element? In this case, the entire document, except perhaps for a few comments and processing instructions that precede or follow the root element, is filtered out. The resulting document does not have a root element. It is thus not well-formed!

Believe it or not, there is no prohibition against this. XML documents must be well-formed, but the only normative definition of an XML document is as a sequence of characters. Here we no longer have a sequence of characters. Instead we have a sequence of method invocations. If those method invocations are coming straight from a parsed XML document, then there are certain guarantees of well-formedness such as there being a single root element. However, if you’re a little further removed from the actual XML document, these guarantees no longer hold. Indeed filters can do considerably weirder things like passing along startElement() calls but blocking the corresponding endElement() calls or passing non-whitespace characters to the ignorableWhiteSpace() method. If your applications have reason to do things like this, go ahead; but document the filter’s behavior exhaustively and be very careful about the client applications that receive data from your filter. Filters can very easily violate a handler’s implicit preconditions with predictably disastrous results.

Filtering attributes

Attributes can be filtered very straightforwardly in the startElement() method. Unlike element content, the complete set of attributes for any element is completely available a single method call as an Attributes object. Thus it’s easy to read through the list and respond appropriately without building complicated data structures to maintain state between method invocations.

The only mildly tricky part is that the Attributes interface is read-only. It provides numerous getter methods but no corresponding setter methods. Thus if you want to do anything other than forward the original Attributes object along unchanged, you’ll need to create your own object that implements the Attributes interface and is mutable. While doing so would be a simple matter of programming, most of the time it’s easier to use the class SAX provides for this purpose, org.xml.sax.helpers.AttributesImpl, summarized in Example 8.10. This class implements the Attributes interface and adds methods for copying existing Attributes objects and adding attributes to and deleting attributes from the list. This class is quite useful when writing filters that process attributes.

Example 8.10. The AttributesImpl helper class

package org.xml.sax.helpers;

public class AttributesImpl implements Attributes {

  public AttributesImpl();
  public AttributesImpl(Attributes atts);

  public int      getLength();
  public String getURI(int index);
  public String getLocalName(int index);
  public String getQualifiedName(int index);
  public String getType(int index);
  public String getValue(int index);
  public int      getIndex(String uri, String localName);
  public int      getIndex(String qualifiedName);
  public String getType(String uri, String localName);
  public String getType(String qualifiedName);
  public String getValue(String uri, String localName);
  public String getValue(String qualifiedName);

  public void clear();
  public void setAttributes(Attributes atts);
  public void addAttribute(String uri, String localName,
   String qualifiedName, String type, String value);
  public void setAttribute(int index, String uri,
   String localName, String qualifiedName, String type,
   String value);
  public void removeAttribute(int index);
  public void setURI(int index, String uri);
  public void setLocalName(int index, String localName);
  public void setQualifiedName(int index, String qualifiedName);
  public void setType(int index, String type);
  public void setValue(int index, String value);

}

As an example, let’s extend the pure XHTML filter of Example 8.9 to remove non-XHTML attributes such as xlink:href from XHTML elements. XHTML attributes are always in no namespace, so we can just drop out any attributes that are in a namespace. The only exception to this are the three xml: attributes: xml:lang, xml:space, and xml:base. These are in the http://www.w3.org/XML/1998/namespace namespace, conveniently represented in SAX by the named constant NamespaceSupport.XMLNS. The revised startElement() method would be as follows:

  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {

    namespaces.push(namespaceURI);
    if (namespaceURI.equals(XHTML_NAMESPACE)) {

      AttributesImpl newAttributes = new AttributesImpl();
      // copy only those attributes that are not in a namespace
      // and do not declare a non-XHTML namespace
      for (int i = 0; i < atts.getLength(); i++) {
        if (atts.getURI(i).equals("")
            || atts.getURI(i).equals(NamespaceSupport.XMLNS)) {
          if (!atts.getQName(i).startsWith("xmlns:")) {
            if (!atts.getQName(i).equals("xmlns")
                || atts.getValue(i).equals(XHTML_NAMESPACE)) {
              newAttributes.addAttribute(atts.getURI(i),
                                         atts.getLocalName(i),
                                         atts.getQName(i),
                                         atts.getType(i),
                                         atts.getValue(i));
            }
          }
        }
      }

      parent.startElement(namespaceURI, localName, qualifiedName,
       newAttributes);
    }
    // else do nothing

  }

The rest of the methods are the same as in Example 8.9, so I won’t repeat them here.

Filters that add content

So far I’ve demonstrated filters that simply removed particular tags, elements, and attributes from the parsed document. However, it’s equally easy to develop filters that insert new content or replace existing content. For example, it’s easy to imagine a filter that converts transitional XHTML to strict XHTML. Such a filter would have to replace deprecated elements like <font size="-1"> with their strict equivalents like <span style="font-size: small">. The details of listing all the tags that need to be fixed are a little tedious, but the algorithm for doing this is not particularly difficult.

More simply, we can write a filter that modifies just one element. Example 8.11 is such a filter. It replaces rddl:resource elements with a simple table. The various attributes of the resource are mapped to different parts of the table. In particular, a rddl:resource like this:

<rddl:resource
        id="note-xlink2rdf"
        xlink:title="W3C NOTE XLink2RDF"
        xlink:role="http://www.w3.org/TR/html4/"
        xlink:arcrole="http://www.rddl.org/purposes#reference"
        xlink:href="http://www.w3.org/TR/xlink2rdf/"
        >
      <li>
        <a href="http://www.w3.org/TR/xlink2rdf/">
          W3C Note Harvesting RDF Statements from XLinks
        </a>
      </li>
</rddl:resource>

will turn into an XHTML table that looks like this:

<table id="note-xlink2rdf">
  <caption>W3C NOTE XLink2RDF</caption>
  <tr><td>Role: </td><td>http://www.w3.org/TR/html4/</td></tr>
  <tr><td>Arcrole: </td><td>http://www.rddl.org/purposes#reference</td></tr>
  <tr><td>Href: </td><td><a href="http://www.w3.org/TR/xlink2rdf/">
   http://www.w3.org/TR/xlink2rdf/</a></td></tr>
  <tr>
    <td colspan="2">
      <li>
        <a href="http://www.w3.org/TR/xlink2rdf/">
          W3C Note Harvesting RDF Statements from XLinks
        </a>
      </li>
    </td>
  </tr>
</table>

This means that a single <rddl:resource> start-tag must create several elements and tags in the output document. There’s no guarantee of a one-to-one mapping between elements in the original and filtered content.

Example 8.11. Changing one element into another

import org.xml.sax.*;
import org.xml.sax.helpers.*;

public class ResourceToTable implements ContentHandler {

  public final static String RDDL_NAMESPACE
   = "http://www.rddl.org/";
  public final static String XHTML_NAMESPACE
   = "http://www.w3.org/1999/xhtml";
  public final static String XLINK_NAMESPACE
   = "http://www.w3.org/1999/xlink";

  private ContentHandler parent;

  public ResourceToTable(ContentHandler parent) {
    this.parent = parent;
  }

  // Replace <rddl:resource> start-tags with the beginning of
  // a table. Move the attributes into table cells.
  // Make the content of the element the last table row.
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {

    if (localName.equals("resource")
        && RDDL_NAMESPACE.equals(namespaceURI)) {

      String hrefAtt    = atts.getValue(XLINK_NAMESPACE, "href");
      String roleAtt    = atts.getValue(XLINK_NAMESPACE, "role");
      String arcroleAtt
       = atts.getValue(XLINK_NAMESPACE, "arcrole");
      String titleAtt   = atts.getValue(XLINK_NAMESPACE, "title");

      // We need to copy the id, xml:lang, and xml:base
      // attributes, if any, from the rddl:resource element to
      // the XHTML table element
      AttributesImpl tableAttributes = new AttributesImpl();

      String id = atts.getValue("id");
      if (id != null) {
        tableAttributes.addAttribute("", "id", "id", "ID", id);
      }

      String lang = atts.getValue("xml:lang");
      if (lang != null) {
        tableAttributes.addAttribute(NamespaceSupport.XMLNS,
         "lang", "xml:lang", "NMTOKEN", lang);
      }

      String base = atts.getValue("xml:base");
      // xml:base is not legal in XHTML. Here I just drop it, but
      // it would be preferable to use it to resolve URLs in the
      // XHTML document before passing them along

      tableAttributes.addAttribute("", "border", "border",
       "NMTOKEN", "1");
      parent.startElement(
       XHTML_NAMESPACE, "table", "table", tableAttributes);

      Attributes noAtts = new AttributesImpl();
      if (titleAtt != null) {
        parent.startElement(
         XHTML_NAMESPACE, "caption", "caption", noAtts);
        characters(titleAtt.toCharArray(), 0, titleAtt.length());
        endElement(XHTML_NAMESPACE, "caption", "caption");
      }

      if (roleAtt != null) {
        parent.startElement(XHTML_NAMESPACE, "tr", "tr", noAtts);
        parent.startElement(XHTML_NAMESPACE, "td", "td", noAtts);
        String role = "Role: ";
        characters(role.toCharArray(), 0, role.length());
        endElement(XHTML_NAMESPACE, "td", "td");
        parent.startElement(XHTML_NAMESPACE, "td", "td", noAtts);
        characters(roleAtt.toCharArray(), 0, roleAtt.length());
        endElement(XHTML_NAMESPACE, "td", "td");
        endElement(XHTML_NAMESPACE, "tr", "tr");
      }

      if (arcroleAtt != null) {
        String arcrole = "Arcrole: ";
        parent.startElement(XHTML_NAMESPACE, "tr", "tr", noAtts);
        parent.startElement(XHTML_NAMESPACE, "td", "td", noAtts);
        characters(arcrole.toCharArray(), 0, arcrole.length());
        endElement(XHTML_NAMESPACE, "td", "td");
        parent.startElement(XHTML_NAMESPACE, "td", "td", noAtts);
        characters(
         arcroleAtt.toCharArray(), 0, arcroleAtt.length());
        endElement(XHTML_NAMESPACE, "td", "td");
        endElement(XHTML_NAMESPACE, "tr", "tr");
      }

      if (hrefAtt != null) {
        String href = "href: ";
        AttributesImpl hrefAtts = new AttributesImpl();
        hrefAtts.addAttribute("", "href", "href", "CDATA", href);
        parent.startElement(XHTML_NAMESPACE, "tr", "tr", noAtts);
        parent.startElement(XHTML_NAMESPACE, "td", "td", noAtts);
        characters(href.toCharArray(), 0, href.length());
        endElement(XHTML_NAMESPACE, "td", "td");
        parent.startElement(XHTML_NAMESPACE, "td", "td", noAtts);
        parent.startElement(XHTML_NAMESPACE, "a", "a", hrefAtts);
        characters(hrefAtt.toCharArray(), 0, hrefAtt.length());
        endElement(XHTML_NAMESPACE, "a", "a");
        endElement(XHTML_NAMESPACE, "td", "td");
        endElement(XHTML_NAMESPACE, "tr", "tr");
      }

      // Now open the contents of the element
      parent.startElement(XHTML_NAMESPACE, "tr", "tr", noAtts);
      AttributesImpl colspanAtts = new AttributesImpl();
      colspanAtts.addAttribute(
       "", "colspan", "colspan", "CDATA", "2");
      parent.startElement(
       XHTML_NAMESPACE, "td", "td", colspanAtts);

    }
    else { // pass the element along
      parent.startElement(namespaceURI, localName, qualifiedName,
       atts);
    }

  }

  // Replace </rddl:resource> end-tags with the end of a table.
  public void endElement(String namespaceURI, String localName,
   String qualifiedName) throws SAXException {

    if (localName.equals("resource")
        && RDDL_NAMESPACE.equals(namespaceURI)) {
      parent.endElement(XHTML_NAMESPACE, "td", "td");
      parent.endElement(XHTML_NAMESPACE, "tr", "tr");
      parent.endElement(XHTML_NAMESPACE, "table", "table");
    }
    else {
      parent.endElement(namespaceURI, localName, qualifiedName);
    }

  }

  // Methods that pass data along unchanged:
  public void startDocument() throws SAXException {
    parent.startDocument();
  }

  public void setDocumentLocator(Locator locator) {
    parent.setDocumentLocator(locator);
  }

  public void endDocument() throws SAXException {
    parent.endDocument();
  }

  public void characters(char[] text, int start, int length)
   throws SAXException {
    parent.characters(text, start, length);
  }

  public void ignorableWhitespace(char[] text, int start,
   int length) throws SAXException {
    parent.ignorableWhitespace(text, start, length);
  }

  public void processingInstruction(String target, String data)
   throws SAXException {
    parent.processingInstruction(target, data);
  }

  public void startPrefixMapping(String prefix, String uri)
   throws SAXException {
    parent.startPrefixMapping(prefix, uri);
  }

  public void endPrefixMapping(String prefix)
   throws SAXException {
    parent.endPrefixMapping(prefix);
  }

  public void skippedEntity(String name)
   throws SAXException {
    parent.skippedEntity(name);
  }

}

Figure 8.5 shows the end of the RDDL specification before this filter is applied. Figure 8.6 shows the end of the RDDL specification after this filter is applied (by passing the filtered source through an XMLWriter and saving the result in a text file.) You can see that the tables have been added as specified.

Figure 8.5. The end of the RDDL specification as normally presented

Figure 8.6. The end of the RDDL specification after rddl:resource elements are replaced by small tables

Filters vs. Transforms

By this point, some readers will have noticed that all these operations could perhaps have been more easily implemented in XSLT. Since XSLT is Turing complete, there really isn’t a lot you can do with a SAX filter that you can’t do with XSLT. The reason you choose one vs. the other is mostly a matter of convenience for the problem at hand. One factor in SAX’s favor is that it is much more efficient when the input or output can be streamed. Unlike XSLT, SAX does not need to read the entire document into memory before the transformation can be performed. The flip side of this is that filters only work well for localized transformations where the output in one part of the document doesn’t depend heavily on the input in another part of the document, especially a part of the document that comes later. For these cases, XSLT is preferable.

The second big advantage to using SAX filters instead of XSLT is that SAX gives you easy access to the full power of Java. For instance a SAX filter can read a book ISBN number and look up the current price of that book at amazon.com to insert as the value of a price attribute it adds to the element. Pure XSLT can’t do this. Of course, you could write an extension function in Java to add this feature to XSLT, so it really is mostly a question of which approach feels more natural to the developers within their particular system and for their particular problem.

[1] In fact, it probably should have been written as a subclass of TransparentFilter that overrode this one method, except that I’ve discovered that whenever I use inheritance to extend one example from an earlier one I get daily e-mail from readers asking me where the missing methods are. The next section will introduce a standard class you can extend to avoid repeating all these methods.

Prev | Up | Next
The XMLFilter interface | Home | The XMLFilterImpl Class
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified December 02, 2001


The XMLFilterImpl Class
Prev Chapter 8. SAX Filters Next

The XMLFilterImpl Class

SAX includes an adapter class you can subclass to build these sorts of two-way filters, org.xml.sax.helpers.XMLFilterImpl. Its general design is similar to what I’ve shown above. However, it implements all the relevant interfaces in one class:

public class XMLFilterImpl implements XMLFilter,
 EntityResolver, DTDHandler, ContentHandler, ErrorHandler

When the various setter methods like setContentHandler() and setErrorHandler() in this class are invoked, the handler is stored in a private field. For example, here’s the setContentHandler() method:

    public void setContentHandler (ContentHandler handler)
    {
        contentHandler = handler;
    }

When the parse() method is called, it swaps out all the installed handlers for the XMLFilterImpl object itself:

    private void setupParse ()
    {
        if (parent == null) {
          throw new NullPointerException("No parent for filter");
        }
        parent.setEntityResolver(this);
        parent.setDTDHandler(this);
        parent.setContentHandler(this);
        parent.setErrorHandler(this);
    }

When the parent parser calls back to the ContentHandler methods, the XMLFilterImpl passes the call back to the original ContentHandler object stored in the contentHandler field. For example, here’s the startElement() method:

    public void startElement (String uri, String localName, String qName,
      Attributes atts)
      throws SAXException
    {
      if (contentHandler != null) {
        contentHandler.startElement(uri, localName, qName, atts);
      }
    }

The other callback methods are similar. Thus by default, XMLFilterImpl doesn’t filter anything, much like the earlier TransparentFilter example. However, you can subclass this class and override those methods where you want to change the data passed back. You pass your changed data by invoking the usual callback methods in this class. Since you may have overridden the relevant methods in a subclass, you may need to use super to access the methods in XMLFilterImpl directly.

For example, the startElement() method in Example 8.12 adds an id attribute to every element that doesn’t already have one, and then passes that modified element on to the underlying content handler to do whatever it needs to do.

Example 8.12. A subclass of XMLFilterImpl

import org.xml.sax.*;
import org.xml.sax.helpers.*;
import java.util.*;

public class IDFilter extends XMLFilterImpl {

  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {

    boolean hasID = false;
    for (int i = 0; i < atts.getLength(); i++) {
      if (atts.getQName(i).equalsIgnoreCase("id") ||
       atts.getType(i).equals("ID")) {
         hasID = true;
         ids.add(atts.getValue(i));
         break;
      }
    }

    if (!hasID) {
      AttributesImpl newAttributes = new AttributesImpl(atts);
      String idValue = makeID();
      newAttributes.addAttribute("", "id", "id", "ID", idValue);
      atts = newAttributes;
    }
    super.startElement(namespaceURI, localName, qualifiedName,
     atts);

  }

  // need to track which IDs we've already used, including IDs
  // that were included in the document
  int id = 1;
  private Set ids; // requires Java 1.2

  public void startDocument() {
    // reinitialize id list for each document
    ids = new HashSet();
    id = 1;
  }

  // Generate an ID that hasn't been used yet
  private String makeID() {

    while (ids.contains("_" + id)) id++;
    ids.add("_" + id);
    return "_" + id;

  }

}

You’ll notice that this code is much shorter and simpler than the programs that implemented XMLFilter directly. There’s a lot of code inside XMLFilterImpl you can reuse without a lot of thought. When subclassing XMLFilterImpl, you only need to override the methods that implement the filter. The remaining methods can be left to the superclass. In fact, it is so much easier to use XMLFilterImpl rather than XMLFilter that almost all real-world filters are based on XMLFilterImpl. A few books even ignore the existence of the XMLFilter interface completely. I mostly covered it here because I spent a lot of time being confused by XMLFilter since I didn’t realize how much more XMLFilterImpl does. It is not just an implementation of the XMLFilter interface.

Since XMLFilterImpl is still an XMLReader, the client application uses it like it would use any other XMLReader, by setting handlers, features, and properties and then parsing documents. The only difference is that the client application needs to pass an actual parser object to the setParent() method before doing anything else.

Here’s the beginning of the output from when I used IDFilter and FilterTester on the RDDL specification, after the usual adjustments for line length. The initial doc processing instruction is an artifact of the XMLWriter class.

% java -Dorg.xml.sax.driver=gnu.xml.aelfred2.XmlReader
  FilterTester http://www.rddl.org/ IDFilter
<?doc type="doctype" role="title" {Resource Directory Description Language 1.0 } ?>
<html xml:lang="en" xml:base="http://www.rddl.org/"
 version="-//XML-DEV//DTD XHTML RDDL 1.0//EN" id="_1"
 xmlns="http://www.w3.org/1999/xhtml">
<head profile="" id="_2">
      <title id="_3">
      XML Resource Directory Description Language (RDDL)</title>
<link href="xrd.css" type="text/css" rel="stylesheet"
 id="_4"></link>
</head>
<body id="_5">
<h1 id="_6">Resource Directory Description Language (RDDL)</h1>
<div class="head" id="_7">
<p id="_8">This Version:
<a href="http://www.openhealth.org/RDDL/20010305" id="_9">
March 5, 2001</a></p>
…

Prev | Up | Next
Content Filters | Home | Parsing non-XML Documents
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified December 02, 2001


Parsing non-XML Documents
Prev Chapter 8. SAX Filters Next

Parsing non-XML Documents

One of the neatest tricks that a SAX filter can perform is presenting something that isn’t an XML document as if it were one. The client application doesn’t need to know that what it’s parsing is really a tab-delimited text file or a relational database if the filter hides all those details and just presents a view of that source as a well-formed XML document.

There are several advantages to using this approach as opposed to actually converting the source. For one, it’s a lot more dynamically adaptable to changing data. For another, it’s easier to integrate into existing XML processing chains. A small advantage is that you don’t need to worry about escaping illegal characters like < and & because these would normally be unescaped before the parser passed them back to the client application.

For example, JDBC makes it easy to write a SAX filter that puts an XML face on a SQL table. The parse() method reads from a JDBC ResultSet which it annotates with tags and attributes by firing the appropriate events. The entire ResultSet becomes a single root table element. Each record is presented in a record element. This record element contains one field element for each field. Each field element contains the value of the field as text. It also has an xsi:type attribute identifying the type of the field and a name attribute giving the name of the field. An alternate approach would use the SQL field names as the element names rather than the generic field with a name attribute. Example 8.13 demonstrates.

Example 8.13. Accessing databases through SAX

import org.xml.sax.*;
import org.xml.sax.helpers.*;
import java.sql.*;
import java.io.*;

public class DatabaseFilter extends XMLFilterImpl {

  private Connection connection;

  // The string passed to the constructor must be a JDBC URL that
  // contains all necessary information for connecting to the
  // database such as host, port, username, password, and
  // database name. For example,
  // jdbc:mysql://host:port]/dbname?user=username&password=pass
  // The driver should have been loaded before this method is
  // called
  public DatabaseFilter(String jdbcURL) throws SQLException {
    connection = DriverManager.getConnection(jdbcURL);
  }

  public void parse(InputSource in) throws SAXException,
   IOException, UnsupportedOperationException {
    throw new UnsupportedOperationException(
     "Can't read a database from an InputStream or Reader"
    );
  }

  public void parse(String selectQuery)
   throws SAXException, IOException {

    try {
      Statement statement = connection.createStatement();
      ResultSet data = statement.executeQuery(selectQuery);
      ResultSetMetaData metadata = data.getMetaData();
      int numFields = metadata.getColumnCount();

      Attributes emptyAttributes = new AttributesImpl();
      startElement("", "table", "table", emptyAttributes);

      while (data.next()) {
        startElement("", "record", "record", emptyAttributes);
        for (int field = 1; field <= numFields; field++) {
          AttributesImpl fieldAtts = new AttributesImpl();
          int type = metadata.getColumnType(field);
          String typeName = getSchemaType(type);
          fieldAtts.addAttribute(
           "http://www.w3.org/2001/XMLSchema-instance", "type",
           "xsi:type", "NMTOKEN", typeName);
          String name = metadata.getColumnName(field);
          fieldAtts.addAttribute(
           "", "name", "name", "NMTOKEN", name);

          // Convert nulls to empty elements with xsi:nil="true"
          Object value = data.getObject(field);
          if (value == null) { // null value in database
            fieldAtts.addAttribute(
             "http://www.w3.org/2001/XMLSchema-instance", "nil",
             "xsi:nil", "NMTOKEN", "true");
            startElement("", "field", "field", fieldAtts);
            endElement("", "field", "field");
          }
          else { // non-null value
            startElement("", "field", "field", fieldAtts);
            convertToXML(data, field, type);
            endElement("", "field", "field");
          }
        }
        endElement("", "record", "record");
      }
      endElement("", "table", "table");
      statement.close();
    }
    catch (SQLException e) {  // convert exception type
      throw new SAXException(e);
    }

  }

  // I want the XML document to store values in the standard W3C
  // XML Schema Language forms. This requires certain conversions
  // depending on the type of the data
  private void convertToXML(ResultSet data, int field, int type)
   throws SQLException, SAXException {

    switch (type) {
      case Types.BINARY:
      case Types.VARBINARY:
      case Types.LONGVARBINARY:
        hexEncode(data.getBinaryStream(field));
        break;
      case Types.BLOB:
        Blob blob = data.getBlob(field);
        hexEncode(blob.getBinaryStream());
        break;
      case Types.CLOB:
        Clob clob = data.getClob(field);
        Reader r = clob.getCharacterStream();
        char[] text = new char[1024];
        int numRead;
        try {
          while ((numRead = r.read(text, 0, 1024)) != -1) {
            escapeText(text, 0, numRead);
            characters(text, 0, numRead);
          }
        }
        catch (IOException e) {
          throw new SAXException("Read from CLOB failed", e);
        }
        break;
      case Types.ARRAY:
        Array array = data.getArray(field);
        writeArray(array);
        break;
      default: // All other types can be handled as strings
        Object o = data.getObject(field);
        if (o == null) return;
        String s = o.toString();
        char[] value = s.toCharArray();
        escapeText(value, 0, value.length);
        characters(value, 0, value.length);
    }

  }

  private void hexEncode(InputStream in)
   throws SQLException, SAXException {

    try {
      int octet;
      while ((octet = in.read()) != -1) {
        StringWriter out = new StringWriter(2);
        if (octet < 16) out.write('0');
        out.write(Integer.toHexString(octet));
        char[] text = out.toString().toCharArray();
        characters(text, 0, 2);
      }
    }
    catch (IOException e) {
      throw new SAXException(e);
    }

  }

  // String types may contain C0 control characters that are
  // not legal in XML. I convert these to the Unicode replacement
  // character 0xFFFD
  private void escapeText(char[] text, int start, int length) {
    for (int i = start; i < length; i++) {
        text[i] = escapeChar(text[i]);
    }
  }

  private char escapeChar(char c) {
    if (c >= 0x20) return c;
    else if (c == '\n') return c;
    else if (c == '\r') return c;
    else if (c == '\t') return c;
    return '\uFFFD';
  }

  private void writeArray(Array array)
   throws SQLException, SAXException {

    ResultSet data = array.getResultSet();
    int type = array.getBaseType();
    String typeName = getSchemaType(type);

    while (data.next()) {
      AttributesImpl fieldAtts = new AttributesImpl();
      fieldAtts.addAttribute(
        "http://www.w3.org/2001/XMLSchema-instance", "type",
        "xsi:type", "NMTOKEN", typeName);
      startElement("", "component", "component", fieldAtts);
      convertToXML(data, 2, type);
      endElement("", "component", "component");
    }

  }

  public static String getSchemaType(int type) {

    switch (type) {
      case Types.ARRAY:         return "array";
      case Types.BIGINT:        return "xsd:long";
      case Types.BINARY:        return "xsd:hexBinary";
      case Types.BIT:           return "xsd:boolean";
      case Types.BLOB:          return "xsd:hexBinary";
      case Types.CHAR:          return "xsd:string";
      case Types.CLOB:          return "xsd:string";
      case Types.DATE:          return "xsd:date";
      case Types.DECIMAL:       return "xsd:decimal";
      case Types.DOUBLE:        return "xsd:double";
      case Types.FLOAT:         return "xsd:decimal";
      case Types.INTEGER:       return "xsd:int";
      case Types.JAVA_OBJECT:   return "xsd:string";
      case Types.LONGVARBINARY: return "xsd:hexBinary";
      case Types.LONGVARCHAR:   return "xsd:string";
      case Types.NUMERIC:       return "xsd:decimal";
      case Types.REAL:          return "xsd:float";
      case Types.REF:           return "xsd:IDREF";
      case Types.SMALLINT:      return "xsd:short";
      case Types.STRUCT:        return "struct";
      case Types.TIME:          return "xsd:time";
      case Types.TIMESTAMP:     return "xsd:dateTime";
      case Types.TINYINT:       return "xsd:byte";
      case Types.VARBINARY:     return "xsd:hexBinary";
                                // most general type
      default:                  return "xsd:string";
    }

  }

  // Warn clients that this filter does not receive its events
  // from another XML parser
  public void setParent(XMLReader parent)
   throws UnsupportedOperationException {
    throw new UnsupportedOperationException(
     "A DatabaseFilter reads from an underlying SQL database;"
     + " not an underlying XML parser"
    );
  }

}

The trickiest part of this design was not the XML output. It was figuring out how to pass in the database connection parameters and the SQL queries. SUN has defined a JDBC URL. However, such a URL only indicates the database, username, and password to access. It does not go all the way down to the level of the SQL query, so it could not be used as a system ID. I chose instead to pass in the database connection parameters through constructors and the SQL query as a system ID string passed to the parse() method. This is not the customary URI system ID, but since this will only be used in this program and not directly in XML documents, this doesn’t cause any major problems. Nonetheless, I changed the name of the formal argument to the parse() method from systemID to SQLQuery to try to make this more obvious. This filter cannot parse InputSource objects because it’s really not possible to read a database from a stream. Thus the parse() method that takes an InputSource as an argument throws an UnsupportedOperationException.

A large part of the logic in this filter involves converting JDBC results into text. Most of the JDBC/SQL types have natural string representations which conveniently match W3C XML Schema Language primitive types. These can be retrieved by calling toString() on the corresponding Java object returned by the JDBC getObject() method. The binary types (BINARY, VARBINARY, LONGVARBINARY, and BLOB) have to be hex encoded first.[2] CLOBs don’t have to be hex encoded, but they aren’t available as a single string, and thus have to be read from a stream too. Finally, arrays and structs require more detailed treatment as a complex type rather than a simple type.

The sample driver program for this filter, shown in Example 8.14, is very similar to earlier driver programs except that it requires the user to specify two arguments on the command line, the JDBC URL giving the connection parameters and the SQL query to execute. Because SQL queries normally contain white space, it must be enclosed in double quotes. A more serious example would reuse the same driver (or filter) for multiple queries.

Example 8.14. A very simple user interface for extracting XML data from a relational database

import org.xml.sax.*;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.*;
import com.megginson.sax.DataWriter;
import java.sql.SQLException;

public class SQLDriver {

  public static void main(String[] args) {

    if (args.length < 2) {
      System.out.println(
       "Usage: java SQLDriver URL query driverClass");
      return;
    }
    String url = args[0];
    String query = args[1];
    String driverClass = "org.gjt.mm.mysql.Driver"; // MySQL
    if (args.length >= 3) driverClass = args[2];

    try {
      // Load JDBC driver
      Class.forName(driverClass).newInstance();
      // Technically, the newInstance() call isn't needed,
      // but the MM.MySQL documentation suggests this to
      // "work around some broken JVMs"

      XMLFilter filter = new DatabaseFilter(url);
      DataWriter out = new DataWriter();
      out.forceNSDecl(
       "http://www.w3.org/2001/XMLSchema-instance", "xsi");
      out.forceNSDecl("http://www.w3.org/2001/XMLSchema", "xsd");
      out.setIndentStep(2);
      filter.setContentHandler(out);
      filter.parse(query);
      out.flush();
    }
    catch (InstantiationException e) {
      System.out.println(driverClass
       + " could not be instantiated");
    }
    catch (ClassNotFoundException e) {
      System.out.println(driverClass + " could not be found");
    }
    catch (Exception e) { // SQL, SAX, and IO
      e.printStackTrace();
      System.out.println(e);
    }

  }

}

Instead of XMLWriter, this driver uses David Megginson’s other public domain writer program, DataWriter, which is designed for record oriented data of this sort and inserts extra white space to prettify the output. Here’s some output from when I ran SQLDriver against one of my databases:

% java -Dorg.xml.sax.driver=gnu.xml.aelfred2.XmlReader
 SQLDriver 'jdbc:mysql://luna.oit.unc.edu/NYWC?user=elharo&password=MyPassword'
 "SELECT LastName, FirstName FROM composers
  WHERE LastName like 'A%'"
<table xmlns:xsd="http://www.w3.org/2001/XMLSchema"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <record>
    <field xsi:type="xsd:string" name="LastName">Anderson</field>
    <field xsi:type="xsd:string" name="FirstName">Beth</field>
  </record>
  <record>
    <field xsi:type="xsd:string" name="LastName">Austin</field>
    <field xsi:type="xsd:string" name="FirstName">Dorothea</field>
  </record>
  <record>
    <field xsi:type="xsd:string" name="LastName">Austin</field>
    <field xsi:type="xsd:string" name="FirstName">Elizabeth</field>
  </record>
  <record>
    <field xsi:type="xsd:string" name="LastName">Ayers</field>
    <field xsi:type="xsd:string" name="FirstName">Lydia</field>
  </record>
</table>

Warning

Depending on platform, inputting the above arguments can be a little tricky. The SQL query contains white space, and the JDBC URL contains characters that are important to the shell like &. You may need to use quote marks to prevent some of these arguments from being interpreted by the shell. Details vary from platform to platform. Of course, this is just a quick hack to demonstrate the filter. A real program would provide a GUI that made these points moot.

[2] Base-64 encoding would be more efficient but I didn’t want to introduce another class library just to do Base-64 encoding.

Prev | Up | Next
The XMLFilterImpl Class | Home | Multihandler adapters
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified December 02, 2001


Multihandler adapters
Prev Chapter 8. SAX Filters Next

Multihandler adapters

The callback interfaces in SAX are reminiscent of the callback interfaces in the AWT like ActionListener and MouseListener. However, there’s one crucial difference in the patterns followed by the SAX callback interfaces and the AWT callback interfaces. The AWT allows you to register multiple listener objects with any one component, whereas SAX limits you to just one listener of a certain type per parser. Filters allow you to parse a document through several listeners in turn. However, sometimes you want to install multiple handlers which are not filters.

For example, the DocBook XML source for the book you’re reading now is parsed multiple times to produce HTML, XHTML, PDF, and several other output formats. Each time a new output format is required, the parser has to reparse the document. However this is really wasted effort. It’s the same document. If it’s well-formed and valid for HTML output, then it’s well-formed and valid for PDF and XHTML output. There’s no need to run the checks anew for each separate output document as long as the input is the same. It makes sense to attach multiple handlers to the parser to avoid wasting time by unnecessarily reparsing the same document.

A properly designed filter can dispatch its events to multiple handlers. The trick is that rather than storing a single ContentHandler/DTDHandler/ErrorHandler the filter stores a list of each. Then each callback method iterates through one of the lists, invoking the method on each of the registered listeners in turn.

When you change the architecture of a class in a subclass as I’m doing here, the naming conventions of the superclass class often don’t make perfect sense in the subclass. In this case, it’s not clear what the various setFooHandler() methods should do. Do they add a new handler to the list? If so what happens when null is passed (the idiom for removing a handler from an XMLReader)? Or should we provide new addFooHandler() and removeFooHandler() methods? And which object from the list should getFooHandler() return? There’s no obvious answer to these questions. Indeed, I changed my mind several times while designing this class and writing this chapter. Here, I chose to keep the interface as similar to the superclass’s as possible. Thus the three setFooHandler() methods add a new handler without replacing the existing handler. However, if null is passed, the entire list is cleared. There’s no easy way to remove a handler or retrieve or delete a specific one from the list. The getFooHandler() method simply returns the first handler in the list. This is not the most flexible design, but it does keep the interface the same as the interface of the superclass. For example, this is how content handlers are stored, added, and removed:

  private List contentHandlers = new ArrayList(2);

  public void setContentHandler(ContentHandler handler) {

    if (handler == null) {
      contentHandlers.clear();
    }
    contentHandlers.add(handler);

  }

  public ContentHandler getContentHandler() {
    if (contentHandlers.isEmpty()) return null;
    return (ContentHandler) contentHandlers.get(0);
  }

However, other approaches are certainly workable.

However the list is filled, the startElement() method would iterate through the list like this:

  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.startElement(namespaceURI, localName,
       qualifiedName, atts);
    }

  }

The other handlers and methods are implemented similarly. Example 8.15 shows the complete class.

Example 8.15. Attaching multiple handlers of the same type to a single parser

import org.xml.sax.*;
import org.xml.sax.helpers.XMLFilterImpl;
import java.util.*;

public class MultiHandlerAdapter extends XMLFilterImpl {

  public MultiHandlerAdapter(XMLReader parent) {
    super(parent);
  }

  List contentHandlers = new ArrayList(2);
  List dtdHandlers = new ArrayList(2);
  List errorHandlers = new ArrayList(2);

  public void setContentHandler(ContentHandler handler) {

    if (handler == null) {
      contentHandlers.clear();
    }
    contentHandlers.add(handler);

  }

  // There's no good way to handle this within the XMLReader
  // interface. I just pick the first in the list.
  public ContentHandler getContentHandler() {
    if (contentHandlers.isEmpty()) return null;
    return (ContentHandler) contentHandlers.get(0);
  }

  public void setDTDHandler(DTDHandler handler) {

    if (handler == null) {
      dtdHandlers.clear();
    }
    dtdHandlers.add(handler);

  }

  public DTDHandler getDTDHandler() {
    if (dtdHandlers.isEmpty()) return null;
    return (DTDHandler) dtdHandlers.get(0);
  }

  public void setErrorHandler(ErrorHandler handler) {

    if (handler == null) {
      errorHandlers.clear();
    }
    errorHandlers.add(handler);

  }

  public ErrorHandler getErrorHandler() {
    if (errorHandlers.isEmpty()) return null;
    return (ErrorHandler) errorHandlers.get(0);
  }

  // ContentHandler implementation
  public void startElement(String namespaceURI, String localName,
   String qualifiedName, Attributes atts) throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.startElement(namespaceURI, localName,
       qualifiedName, atts);
    }

  }

  public void endElement(String namespaceURI, String localName,
   String qualifiedName) throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.endElement(namespaceURI, localName,
       qualifiedName);
    }

  }

  public void characters(char[] text, int start, int length)
   throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.characters(text, start, length);
    }

  }

  public void ignorableWhitespace(char[] text, int start,
   int length) throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.ignorableWhitespace(text, start, length);
    }
  }

  public void processingInstruction(String target, String data)
   throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.processingInstruction(target, data);
    }
  }

  public void skippedEntity(String name)
   throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.skippedEntity(name);
    }
  }

  public void startPrefixMapping(String prefix, String uri)
   throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.startPrefixMapping(prefix, uri);
    }

  }

  public void endPrefixMapping(String prefix)
   throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.endPrefixMapping(prefix);
    }

  }

  public void startDocument() throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.startDocument();
    }

  }

  public void endDocument() throws SAXException {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.endDocument();
    }

  }

  public void setDocumentLocator(Locator locator) {

    Iterator handlers = contentHandlers.iterator();
    while(handlers.hasNext()) {
      ContentHandler handler = (ContentHandler) handlers.next();
      handler.setDocumentLocator(locator);
    }

  }

  // DTDHandler implementation
  public void notationDecl(String name, String publicID,
   String systemID) throws SAXException {

    Iterator handlers = dtdHandlers.iterator();
    while(handlers.hasNext()) {
      DTDHandler handler = (DTDHandler) handlers.next();
      handler.notationDecl(name, publicID, systemID);
    }

  }

  public void unparsedEntityDecl(String name, String publicID,
   String systemID, String notationName) throws SAXException {

    Iterator handlers = dtdHandlers.iterator();
    while(handlers.hasNext()) {
      DTDHandler handler = (DTDHandler) handlers.next();
      handler.unparsedEntityDecl(name, publicID, systemID,
       notationName);
    }

  }

  // ErrorHandler implementation
  public void warning(SAXParseException exception)
   throws SAXException {

    Iterator handlers = errorHandlers.iterator();
    while(handlers.hasNext()) {
      ErrorHandler handler = (ErrorHandler) handlers.next();
      handler.warning(exception);
    }

  }

  public void error(SAXParseException exception)
   throws SAXException {

    Iterator handlers = errorHandlers.iterator();
    while(handlers.hasNext()) {
      ErrorHandler handler = (ErrorHandler) handlers.next();
      handler.error(exception);
    }

  }

  public void fatalError(SAXParseException exception)
   throws SAXException {

    Iterator handlers = errorHandlers.iterator();
    while(handlers.hasNext()) {
      ErrorHandler handler = (ErrorHandler) handlers.next();
      handler.fatalError(exception);
    }

  }

}

Note

MultiHandlerAdapter is a little on the long side. It contains a lot of duplicated code. This is definitely a case where some space could be saved by using templates.

MultiHandlerAdapter only provides the three main callback interfaces: ContentHandler, DTDHandler, and ErrorHandler. I chose not to provide EntityResolver because having more than one of those responding to the same request doesn’t make sense. Each entity must be provided exactly once, not two or three or seventeen times. Potentially, you could register multiple entity resolvers and then iterate through them in sequence until one found the entity you were looking for. However, since order would be very significant in that case, a proper API is much trickier to design.

I also chose not to implement the property based handlers, LexicalHandler and DeclHandler. First, not all parsers support them. Secondly, they are not part of the standard XMLFilterImpl class. Most importantly, adding and removing these objects solely by setting and getting features and properties would be very messy. If you need multiples of one of them, it seems simpler to implement the multi-way dispatching in a LexicalHandler/DeclHandler implementation rather than directly in the filter. A similar scheme could work for ContentHandler, DTDHandler, and ErrorHandler. However, here the filter approach seems the most natural.

Prev | Up | Next
Parsing non-XML Documents | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified December 02, 2001


Summary
Prev Chapter 8. SAX Filters Next

Summary

SAX filters sit between a client application and a parser (XMLReader) and intercept calls from the client application to the parser. SAX filters are instances of implementations of the org.xml.sax.XMLFilter interface, a subinterface of XMLReader. Thus a SAX filter is also a parser, albeit one that receives its data from another XML parser rather than by directly reading an XML document.

The easiest way to write a SAX filter is by subclassing org.xml.sax.helpers.XMLFilterImpl, which implements several interfaces including XMLFilter, ContentHandler, DTDHandler, and ErrorHandler. org.xml.sax.helpers.XMLFilterImpl intercepts calls in both directions, from the client to the parser and the parser to the client. The default behavior of this class is to pass all events along unchanged. By overriding the standard methods of the various handler interfaces, you can change the data a client application receives from the parser. This gives the filter the opportunity to log, modify, block, supplement, or replace each call.

Document well-formedness is normally verified against the original text document by the parser. Most client applications assume the data they receive through SAX is well-formed. For instance, they assume there is only a single root element. However, it’s possible for a filter to violate these assumptions. For instance, the root element start-tag and end-tag could be filtered out while leaving the contents intact. Illegal characters like null and vertical tab could be passed to the characters() method. Unless you know exactly how your handlers will respond to such malformed event streams, you should be careful to make sure that your filters maintain well-formedness, and, if necessary, validity.

Prev | Up | Next
Multihandler adapters | Home | Chapter 9. The Document Object Model
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified November 30, 2001


Chapter 9. The Document Object Model
Prev Next

Chapter 9. The Document Object Model

Table of Contents

The Evolution of DOM
DOM Modules
Application Specific DOMs
Trees
Document nodes
Element nodes
Attribute nodes
Leaf nodes
Non-tree nodes
What is and isn’t in the tree
DOM Parsers for Java
Parsing documents with a DOM Parser
JAXP DocumentBuilder and DocumentBuilderFactory
DOM3 Load and Save
The Node Interface
Node Types
Node Properties
Navigating the tree
Modifying the tree
Utility Methods
The NodeList interface
JAXP Serialization
DOMException
Choosing between SAX and DOM
Summary

The Document Object Model, DOM for short, is an abstract data structure that represents XML documents as trees of nodes. Different interfaces in the org.w3c.dom package represent elements, attributes, parsed character data, comments, and processing instructions. All of these interfaces are sub-interfaces of the common Node interface that provides basic methods for navigating and pruning the tree.

The root of the tree is a Document object that represents a complete well-formed document. A parser reads an XML document from a stream and builds a Document object representing that XML document. The client program calls the methods of Document and the other DOM interfaces to navigate the tree and extract information from the document. Programs can also manipulate the tree in memory to add, delete, move, or change nodes in the tree. Programs can even create completely new documents from scratch in memory which are then written into an XML file.

DOM is defined in the Interface Definition Language (IDL) so that it’s language neutral. DOM bindings exist for most object-oriented languages including Java, JavaScript, C++, Python, and Perl. However, since this is a book about Java, I will deal exclusively with the Java implementation in this chapter.

The Evolution of DOM

The first version of DOM wasn’t an official specification, just the object model that Netscape Navigator 3 and Internet Explorer 3 implemented in their browsers. (Really, these were two different object models since they were only marginally compatible with each other.) This is sometimes called DOM Level 0.

DOM Level 0 only applied to HTML documents and only in the context of JavaScript. Nonetheless, both the usefulness of JavaScript and the growing incompatibility between the two browser object models made it obvious that something more standard was needed. Hence, the W3C launched the W3C DOM Activity and began working on DOM Level 1. DOM1 was an attempt to come out with a specification as quickly as possible that would codify existing practice while achieving some level of compatibility across browsers. Given the constraints the working group was laboring under, DOM1 is a surprisingly good spec. The naming conventions feel a little wrong to a Java programmer, but DOM1 does provides a solid core of functionality that covers maybe 75% of what programmers want to do when processing XML.

DOM Level 2 cleaned up the DOM Level 1 interfaces. The big change was namespace support in the Element and Attr interfaces. In addition, DOM2 added a number of supplementary interfaces for events, traversal, ranges, views, and style sheets. I’ll address these in upcoming chapters. In 2002, all significant XML parsers that support DOM, support DOM Level 2. There’s not a lot of reason to worry about the difference between DOM1 and DOM2. From this point forward, I’m just going to teach DOM2.

DOM Level 3 is visible not far up the road. Parts of it are just beginning to be supported by bleeding edge parsers, most especially Xerces 2. In the core, DOM3 just adds a few missing pieces needed to allow DOM to fully support all Infoset properties. This includes the original encoding and base URI of the document. However, DOM3 will also add some crucial functionality missing from DOM2. In particular, DOM2 doesn’t provide a parser-independent means to create a new Document object, either by parsing a file or by building one from scratch in memory. DOM3 will provide standard ways of doing both. DOM3 is also going to add a lot more support for DTDs and schemas. But despite all its new features and functionality, DOM3 will not replace DOM2. Everything that works today in DOM2 will continue to work the same way in DOM3. DOM3 extends the DOM into new territory, but it doesn’t change what has gone before.

Prev | Up | Next
Summary | Home | DOM Modules
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


DOM Modules
Prev Chapter 9. The Document Object Model Next

DOM Modules

DOM2 is divided into fourteen modules in eight different packages. DOM1 roughly corresponds to the Core and XML modules. The other twelve are new in DOM2.

Core: org.w3c.dom

The basic interfaces that can be used to represent any SGMLish, hierarchical tree-structured document including DOMException, DOMImplementation, DocumentFragment, Document, Node, NodeList, NamedNodeMap, CharacterData, Attr, Element, Text, and Comment.

XML: org.w3c.dom

The additional sub-interfaces of Node just for XML documents including CDATASection, DocumentType, Notation, Entity, EntityReference, and ProcessingInstruction. An HTML DOM might not implement these.

HTML: org.w3c.dom.html

Interfaces designed specifically to represent the parts of an HTML document such as HTMLHtmlElement, HTMLHeadElement, and HTMLParagraphElement. These are all sub-interfaces of the core interfaces like Node and Element. These aren’t normally relevant to processing XML documents. However, if you have a DOM aware HTML parser, you can apply the techniques described in this book to HTML as well as XML.

Views: org.w3c.dom.views

The AbstractView and DocumentView interfaces used to associate different views with one document. For instance, applying two style sheets to one XML document could produce two views. This is not supported by most XML parsers.

StyleSheets: org.w3c.dom.stylesheets

Basic interfaces for representing style sheets including StyleSheet, StyleSheetList, MediaList, LinkStyle, and DocumentStyle.

CSS: org.w3c.dom.css

Interfaces that specifically represent CSS style sheets including CSSStyleSheet, CSSRuleList, CSSRule, CSSStyleRule, CSSMediaRule, CSSFontFaceRule, CSSPageRule, CSSImportRule, CSSCharsetRule, CSSUnknownRule, CSSStyleDeclaration, CSSValue, CSSPrimitiveValue, CSSValueList, RGBColor, Rect, Counter, ViewCSS, DocumentCSS, DOMImplementationCSS, and ElementCSSInlineStyle.

CSS2: org.w3c.dom.css

The CSS2Properties class that provides shortcut methods for setting all the different CSS2 style properties.

Events: org.w3c.dom.events

The interfaces in these classes establish a generic system that allows event listeners to be attached to nodes. Events nodes can respond to user interface events like mouse clicks, structure modification events like document edits, and anything else the implementation wants to support. The next four modules define specific kinds of events. However, applications can define their own as well. In practice, events are more relevant to JavaScript and other browser-based script systems than most XML programs; but they can be useful when an XML document is displayed in a browser or otherwise shown to a user.

UIEvents: org.w3c.dom.events

The UIEvent interface signals when a node represented on the screen in some form of GUI has received the focus, lost the focus, or been activated.

MouseEvents: org.w3c.dom.events

The MouseEvent interface signals when, where, and with which keys pressed the user has clicked the mouse, pressed the mouse button, released the mouse button, moved the mouse, or moved the cursor into or out of an element’s graphical representation.

MutationEvents: org.w3c.dom.events

The MutationEvent interface signals that Cerebro has detected a new mutant of exceptional power. Sorry about that one, I couldn’t help myself. Really it signals that a node has been added, removed, or modified in the document.

HTMLEvents: org.w3c.dom.events

The HTML events module doesn’t define any new interfaces. Instead it uses the base DOMEvent interface to report a dozen events specific to web browsers including load, unload, abort, error, select, change, submit, reset, focus, blur, resize, and scroll.

Traversal: org.w3c.dom.traversal

This package provides simple utility classes for performing common operations on a tree such as walking the entire tree or filtering out nodes that meet certain conditions. It will be discussed in depth in Chapter 12.

Range: org.w3c.dom.ranges

This optional module extends DOM to cover sections of documents that don’t neatly match element boundaries. For instance, it would be useful for indicating the section of text that the user has selected with the mouse. It could also be used for XPointer ranges.

Aside from the core and XML modules, not all DOM implementations support all of these modules, or all parts of those modules they do support. Most Java implementations do support the traversal module. The events module is not uncommon. The range module is occasionally supported. As far as I know, only Xerces supports the HTML module. So far I haven’t found any parsers that support the Views, Style Sheets or CSS modules.

The hasFeature() method in the DOMImplementation interface can tell you if that implementation supports a particular feature. Just pass in the name and version of the feature you’re looking for. For DOM Level 2 the version is "2.0".

public boolean hasFeature(String name, String version);

Besides the standard modules, implementations and application-specific DOMs may define additional feature name strings. These non-standard features use a reversed domain name, much like a Java package name, to clearly indicate who is responsible for the feature. For instance, SVG 1.0 uses the feature name "org.w3c.dom.svg" and the version number "1.0" to indicate support for some part of the SVG DOM. It uses the feature strings "org.w3c.dom.svg.static", "org.w3c.dom.svg.animation", or "org.w3c.dom.svg.dynamic" to indicate support for specific parts of the SVG DOM.

Different DOM implementations use different concrete classes to implement the standard interfaces. For example, in Xerces, the org.apache.xerces.dom.DOMImplementationImpl singleton class implements the DOMImplementation interface. In the Oracle XML Parser for Java, XMLDOMImplementation implements the DOMImplementation interface. This class has a simple no-args constructor. Example 9.1 uses this class and constructor to check for the standard features in Oracle.

Example 9.1. Which modules does Oracle support?

import oracle.xml.parser.v2.XMLDOMImplementation;
import org.w3c.dom.DOMImplementation;

public class OracleModuleChecker {

  public static void main(String[] args) {

    // parser dependent
    DOMImplementation implementation = new XMLDOMImplementation();
    String[] features = {"Core", "XML", "HTML", "Views",
     "StyleSheets", "CSS", "CSS2", "Events", "UIEvents",
     "MouseEvents", "MutationEvents", "HTMLEvents", "Traversal",
     "Range"};

    for (int i = 0; i < features.length; i++) {
      if (implementation.hasFeature(features[i], "2.0")) {
        System.out.println("Oracle supports " + features[i]);
      }
      else {
        System.out.println("Oracle does not support "
         + features[i]);
      }
    }

  }

}

Here’s the output from version 9.0.1.1.0A Production of the Oracle XML Parser for Java. You see that Oracle only supports the XML, events, traversal, and range modules. The reported lack of support for core is almost certainly just an oversight. The "Core" feature string was only added in the final draft of DOM 2, and was not present in earlier drafts. The Core module is a prerequisite for all the other modules. It’s hard to believe Oracle really doesn’t support it.

D:\books\XMLJAVA\examples\09>java OracleModuleChecker
Oracle does not support Core
Oracle supports XML
Oracle does not support HTML
Oracle does not support Views
Oracle does not support StyleSheets
Oracle does not support CSS
Oracle does not support CSS2
Oracle supports Events
Oracle does not support UIEvents
Oracle does not support MouseEvents
Oracle does not support MutationEvents
Oracle does not support HTMLEvents
Oracle supports Traversal
Oracle supports Range
Prev | Up | Next
Chapter 9. The Document Object Model | Home | Application Specific DOMs
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified December 20, 2001


Application Specific DOMs
Prev Chapter 9. The Document Object Model Next

Application Specific DOMs

A number of XML applications have built useful application-specific DOMs by extending the standard DOM interfaces. XML applications with their own custom DOMs include HTML and XHTML, the Wireless Markup Language (WML), Scalable Vector Graphics (SVG), and MathML.

Where the generic DOM would use an Element object, a WML-specific DOM might use a WMLOptionElement or a WMLPElement or a WMLPostfieldElement object, as appropriate for the actual type of element it represents. These custom subclasses and subinterfaces have all the methods and properties of the standard interfaces, as well as other methods and properties appropriate only for their type. For example, A WML p element has align, mode, and xml:lang attributes, like this:

<p align="center" mode="wrap" xml:lang="en">
  Hello!
</p>

Therefore, the WMLPElement interface has getter and setter methods for those three attributes:

public void setMode(String mode);
public void setAlign(String align);
public void setXMLLang(String lang);
public String getMode();
public String getAlign();
public String getXMLLang();

An application specific DOM can enforce application specific rules such as “The mode attribute must have one of the values wrap or nowrap,” though currently this isn’t very common.

Of course, since WMLPElement extends Element which extends Node, it also has the usual methods of any DOM node. When processing a WML document, you can use the generic DOM interfaces if you prefer or you can use the more specific WML subclasses and subinterfaces.

The big issue for most of the application specific DOMs is parser support. To read these documents, you not only need a custom DOM. You also need a custom parser that knows how to generate the application specific DOM. That’s a little harder to come by. With some effort, the Xerces DOM parser can be configured to produce HTML DOM Document objects for well-formed HTML and XHTML. The XML Apache Project’s open source Batik includes an SVG parser that can produce SVG DOM Document objects. For other application specific DOMs, the pickings are a little slim right now.

Creating new WML/SVG/MathML/etc. documents in a particular vocabulary using an application specific DOM is somewhat easier. However, you do still need a concrete implementation of that DOM’s abstract interfaces. Xerces includes HTML and WML implementations. Batik includes one for SVG. For other models, you’ll have to roll your own.

Prev | Up | Next
DOM Modules | Home | Trees
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 08, 2002


Trees
Prev Chapter 9. The Document Object Model Next

Trees

DOM is based on an implicit data model, which is similar to but not quite the same as the data models used by other XML technologies like XPath, the XML Infoset, and SAX. Before we delve too deeply into the nitty-gritty details of the DOM API, it’s helpful to have a higher level understanding of just what DOM thinks an XML document is.

According to DOM, an XML document is a tree made up of nodes of several types. The tree has a single root node, and all nodes in this tree except for root have a single parent node. Furthermore, each node has a list of child nodes. In some cases, this list of children may be empty, in which case the node is called a leaf node.

There can also be nodes that are not part of the tree structure. For instance, each attribute node belongs to one element node but is not considered to be a child of that element. Furthermore, nodes can be removed from the tree or created but not inserted in the tree. Thus a full DOM document is composed of a tree of nodes, various nodes that are somehow associated with other nodes in the tree but are not themselves part of the tree, and a random assortment of disconnected nodes.

DOM trees are not red-black trees, binary trees, B-trees, or any other sort of special-purpose tree. From a data structures point of view, they are just plain-vanilla trees. Recursion works very well on DOM data structures, as it does on any tree. You can use all the techniques you learned in Data Structures 201 for processing trees. Breadth-first search, depth-first search, inorder traversal, preorder traversal, postorder traversal, etc. all work with DOM data structures.

Besides its tree connections, each node has a local name, a namespace URI, and a prefix; though for several kinds of nodes, these may be null. For instance the local name, namespace URI, and prefix of a comment are always null. Each node also has a node name. For an element or attribute, the node name is the prefixed name. For other named things like notations or entities, the node name is the name of the thing. For nodes without names such as text nodes, the node name is the value from the following list matching the node type:

  • #document

  • #comment

  • #text

  • #cdata-section

  • #document-fragment

Finally each node has a string value. For text-ish things like text nodes and comments, this tends to be the text of the node. For attributes, it’s the normalized value of the attribute. For everything else, including elements and documents, the value is null.

DOM divides nodes into twelve types, seven of which can potentially be part of a DOM tree:

  • Document nodes

  • Element nodes

  • Text nodes

  • Attribute nodes

  • Processing instruction nodes

  • Comment nodes

  • Document type nodes

  • Document fragment nodes

  • Notation nodes

  • CDATA section nodes

  • Entity nodes

  • Entity reference nodes

However, of these twelve, the first seven are by far the most important; and a tree built by an XML parser will often contain only the first seven.

Document nodes

Each DOM tree has a single root document node. This node has children. Since all documents have root elements, a document node always has exactly one element node child. If the document has a document type declaration, then it will also have one document type node child. If the document contains any comments or processing instructions before or after the root element, then these will also be child nodes of the document node. The order of all children is maintained. For example, consider the simple XML-RPC document shown in Example 9.2.

Example 9.2. An XML-RPC request document

<?xml version="1.0"?>
<?xml-stylesheet type="text/css" href="xml-rpc.css"?>
<!-- It's unusual to have an xml-stylesheet processing
     instruction in an XML-RPC document but it is legal, unlike
     SOAP where processing instructions are forbidden. -->
<!DOCTYPE methodCall SYSTEM "xml-rpc.dtd">
<methodCall>
  <methodName>getQuote</methodName>
  <params>
    <param>
      <value><string>RHAT</string></value>
    </param>
  </params>
</methodCall>

The document node representing the root of this document has four child nodes in this order:

  • A processing instruction node for the xml-stylesheet processing instruction

  • A comment node for the comment

  • A document type node for the document type declaration

  • An element node for the root methodCall element

The XML declaration (including the version, standalone, and encoding declarations) and the white space between these nodes are not included in the tree. They are not part of the model, and the parser does not include them in the tree it builds.

Element nodes

Each element node has a name, a local name, a namespace URI (which may be null if the element is not in any namespace) and a prefix (which may also be null). It also contains children. For example, consider this value element:

<value><string>RHAT</string></value>

When represented in DOM, it becomes a single element node with the name value. This node has a single element node child for the string element. Thie string element node has a single text node child containing the text RHAT.

Or consider this para element:

<db:para xmlns:db="http://www.example.com/"
  xmlns="http://namespaces.cafeconleche.org/">
  Or consider this <markup>para</markup> element:
</db:para>

In DOM it’s represented as an element node with the name db:para, the local name para, the prefix db, and the namespace URI http://www.example.com/. It has three children:

  • A text node containing the text Or consider this

  • An element node with the name markup, the local name markup, the namespace URI http://namespaces.cafeconleche.org/, and a null prefix.

  • Another text node containing the text element:.

White space is included in text nodes, even if it’s ignorable. For example, consider this methodCall element:

<methodCall>
  <methodName>getQuote</methodName>
  <params>
    <param>
      <value><string>RHAT</string></value>
    </param>
  </params>
</methodCall>

It is represented as an element node with the name methodCall and five child nodes:

  • A text node containing only white space

  • An element node with the name methodName

  • A text node containing only white space

  • An element node with the name params

  • A text node containing only white space

Of course, these element nodes also have their own child nodes.

As well as element and text nodes, an element node can also contain comment and processing instruction nodes. Depending on how the parser behaves, an element node might also contain some CDATA section nodes and/or entity reference nodes. However, many parsers resolve these automatically into their component text and element nodes and do not report them separately.

Attribute nodes

An attribute node has a name, a local name, a prefix, a namespace URI, and a string value. The value is normalized as required by the XML 1.0 specification. That is, entity and character references in the value are resolved, and all white space characters are converted to a single space. If the attribute has any type other than CDATA, then leading and trailing white space is stripped from its value, and all other runs of white space are converted to a single space. An attribute node also has children, all of which are text and entity reference nodes forming the value of the attribute. However, it’s unusual to access these directly instead of by the value.

If a validating parser builds an XML document from a file, then default attributes from the DTD are included in the DOM tree. If the parser supports schemas, then default attributes can be read from the schema as well.

DOM does not provide the type of the attribute as specified by the DTD or schema, or the list of values available for an enumerated type attribute. This is a major shortcoming.

Attributes are not considered to be children of the element they’re attached to. Instead they are part of a separate set of nodes. For example, consider this Quantity element:

<Quantity amount="17" />

This element has no children, but it does have a single attribute with the name amount and the value 17.

Attributes that declare namespaces are not treated specially in DOM. They are reported by DOM parsers like any other attribute. Furthermore, DOM always provides the fully qualified names and namespace URIs for all element and attribute nodes.

Leaf nodes

Only document, element, attribute, entity, and entity reference nodes can have children. The remaining node types are much simpler.

Text nodes

Text nodes contain character data from the document stored as a String. Any characters like 𝄢 from outside Unicode’s Basic Multilingual Plane are represented as surrogate pairs. Characters like & and < that are represented in the document by predefined entity or character references are replaced by the actual characters they represent. If these nodes are written out again into an XML document, these characters need to be reescaped.

When a parser reads an XML document to form a DOM Document, it will put as much text as possible into each text node before being interrupted by non-predefined entities, comments, tags, CDATA section delimiters or other markup. Thus no text node will immediately follow any other text node. There is always an intervening non-text node. However, if a DOM is created or modified in memory, then the client program may divide text between immediately adjacent text nodes, so it’s not guaranteed that each text node contains the maximum possible contiguous run of text, just that this is the case immediately after a document is parsed.

Comment nodes

A comment node has a name (which is always #comment), a string value (the text of the comment) and a parent (the node that contains it). That’s all. For example, consider this comment:

<!-- Don't forget to fix this! -->

The value of this node is Don't forget to fix this! . The white space on either end is included.

Processing instruction nodes

A processing instruction node has a name (the target of the processing instruction), a string value (the data of the processing instruction) and a parent (the node that contains it). That’s all. For example, consider this processing instruction:

<?xml-stylesheet type="text/css" href="xml-rpc.css"?>

The name of this node is xml-stylesheet. The value is type="text/css" href="xml-rpc.css". The white space between the target and the data is not included. White space between the data and the closing ?> is included. Even if the processing instruction uses a pseudo-attribute format like this one does, it is not considered to actually have attributes or children. Its data is just a string that happens to have some equal signs and quote marks in suggestive positions.

CDATA section nodes

A CDATA section node is a special text node that represents the contents of a CDATA section. Its name is #cdata-section. Its value is the text content of the section. For example, consider this CDATA section:

<![CDATA[<?xml-stylesheet type="text/css" href="xml-rpc.css"?>]]>

Its name is #cdata-section and its value is <?xml-stylesheet type="text/css" href="xml-rpc.css"?>.

Entity reference nodes

When a parser encounters a general entity reference such as &AElig; or &copyright_notice;, it may or may not replace it with the entity’s replacement text. Validating parsers always replace entity references. Non-validating parsers may do so at their option.

If a parser does not replace entity references, then the DOM tree will include entity reference nodes. Each entity reference node has a name. If the parser has read the DTD, then you should be able to look up the public and system IDs for this entity reference using the map of entity nodes available on the document type node. Furthermore, the child list of the entity will contain this entity reference’s replacement text. However, if the parser has not read the DTD and resolved external entity references, then the child list may be empty.

If a parser does replace entity references, then the DOM tree may or may not include entity reference nodes. Some parsers resolve all entity reference nodes completely and leave no trace of them in the parsed tree. However, some parsers instead include entity reference nodes in the DOM tree that have a list of children. The child list contains text nodes, element nodes, comment nodes, and so forth representing the replacement text of the entity.

For example, suppose an XML document contains this element:

<para>&AElig;lfred is a very nice XML parser.</para>

If the parser is not resolving entity references, then the para element node contains two children, an entity reference node with the name AElig and a text node containing the text “lfred is a very nice XML parser.” The AElig entity reference node will not have any children.

Now suppose the parser is resolving entity references, and the replacement text for the AElig entity reference is the single ligature character Æ. Then the parser has a choice. It can represent the children of the para element as a single text node containing the full sentence, “Ælfred is a very nice XML parser.” Alternately, it can represent the children of the para element as an entity reference node with the name AElig followed by a text node containing the text, “lfred is a very nice XML parser.” If it chooses the second option, then the AElig entity reference node contains a single read-only text node child containing single ligature character Æ.

DOM never includes entity reference nodes for the five predefined entity references: &amp;, &lt;, &gt;, &apos;, and &quot;. These are simply replaced by their respective characters and included in a text node.

Similarly, character references like &#xA0 and &#160; are not specially represented in DOM as any kind of node. The characters they represent are simply added to the relevant text node.

Document type nodes

A document type node has a name (the name the document type declaration specifies for the root element), a public ID (which may be null), a system ID (required), an internal DTD subset (which may be null), a parent (the document that contains it), and lists of the notations and general entities declared in the DTD. The value of a document type node is always null. For example, consider this document type declaration:

<!DOCTYPE mml:math PUBLIC "-//W3C//DTD MathML 2.0//EN"
 "http://www.w3.org/TR/MathML2/dtd/mathml2.dtd" [
  <!ENTITY % MATHML.prefixed "INCLUDE">
  <!ENTITY % MATHML.prefix "mml">
]>

The name of the corresponding node is mml:math. The public ID is -//W3C//DTD MathML 2.0//EN. The system ID is http://www.w3.org/TR/MathML2/dtd/mathml2.dtd. The internal DTD subset is the complete text between the [ and the ].

Non-tree nodes

There are four kinds of DOM nodes that are part of the document but not the document’s tree: attribute nodes, entity nodes, notation nodes, and document fragment nodes. You’ve already seen that attribute nodes are attached to element nodes, but are not children of those nodes. Entity and notation nodes are available as properties of the document type node. Document fragment nodes are only used when building DOM trees in memory, not when reading them from a parsed file.

Entity nodes

Entity nodes (not to be confused with entity reference nodes) represent the parsed and unparsed entities declared in the document’s DTD. If the parser reads the DTD, then it will attach a map of entity nodes to the document type node. This map is indexed by the entity names. You can use it to match entity reference nodes to entity nodes.

Each entity node has a name and a system ID. It can also have a public ID if one was used in the DTD. Furthermore, if the parser read the entity, then the entity node has a list of children containing the replacement text of the entity. However, these children are read-only and cannot be modified, unlike children of similar type elsewhere in the document. For example, suppose the following entity declaration appears in the document’s DTD:

<!ENTITY AElig "&#x00C6;">

If the parser reads the DTD, then it will create an entity node with the name AElig. This node will have a null public and system ID (because the entity’s purely internal) and one child, a read-only text node containing the single character Æ.

For another example, suppose this entity declaration appears in the document’s DTD:

<!ENTITY Copyright SYSTEM "copyright.xml">

If the parser reads the DTD, then it will create an entity node with the name Copyright, the system ID copyright.xml, and a null public ID. The children of this node will depend on what’s found at the relative URL copyright.xml. Suppose that document contains the following content:

<copyright>
  <year>2002</year>
  <person>Elliotte Rusty Harold</person>
</copyright>

Then the child list of the Copyright entity node will contain a single read-only element child with the name copyright. The element will contain its own read-only element and text node children.

Notation nodes

Notation nodes represent the notations declared in the document’s DTD. If the parser reads the DTD, then it will attach a map of notation nodes to the document type node. This map is indexed by the notation name. You can use it to look up the notation for each entity node that corresponds to an unparsed entity or the notations associated with particular processing instruction targets.

Besides its name, each notation node has a public ID or a system ID, whichever was used to declare it in the DTD. Notation nodes do not have any children. For example, suppose this notation declaration for PNG images were included in the DTD:

<!NOTATION PNG SYSTEM "http://www.w3.org/TR/REC-png">

This would produce a notation node with the name PNG and the system ID http://www.w3.org/TR/REC-png. The public ID would null.

For another example, suppose this notation declaration for TeX documents were included in the DTD:

<!NOTATION TEX PUBLIC
    "+//ISBN 0-201-13448-9::Knuth//NOTATION The TeXbook//EN">

This would produce a notation node with the name TEX and the public ID +//ISBN 0-201-13448-9::Knuth//NOTATION The TeXbook//EN. The system ID would null. (XML doesn’t allow notations to have both public and system IDs.)

Document fragment nodes

The document fragment node is an alternative root node for a DOM tree. It can contain anything an element can contain (i.e. element nodes, text nodes, processing instruction nodes, comment nodes, etc.). A parser will never produce such a node. However, your own programs may create such a node when extracting part of an XML document in order to move it elsewhere.

In DOM, the non-root nodes never exist alone. That is, there’s never a text node or an element node or a comment node that’s not part of a document or a document fragment. They may be temporarily disconnected from the main tree, but they always know which document or fragment they belong to. The document fragment node enables you to work with pieces of a document that are composed of more than one node.

What is and isn’t in the tree

Table 9.1 summarizes the DOM data model with the name, value, parent, and possible children for each kind of node.

Table 9.1. Node properties

Node type | name | value | parent | children
Document | #document | null | null | Comment, Processing Instruction, 1 Element
DocumentType | Root element name specified by the DOCTYPE declaration | null | Document | none
Element | prefixed name | null | Element, Document, or Document fragment | Comment, Processing Instruction, Text, Element, Entity reference, CDATA section
Text | #text | text of the node | Element, Attr, Entity, or Entity reference | none
Attr | prefixed name | normalized attribute value | Element | Text, Entity reference
Comment | #comment | text of comment | Element, Document, or Document fragment | none
Processing Instruction | target | data | Element, Document, or Document fragment | none
Entity Reference | name | null | Element or Document Fragment | Comment, Processing Instruction, Text, Element, Entity reference, CDATA section
Entity | entity name | null | null | Comment, Processing Instruction, Text, Element, Entity Reference, CDATA section
CDATA section | #cdata-section | text of the section | Element, Entity, or Entity reference | none
Notation | notation name | null | null | none
Document fragment | #document-fragment | null | null | Comment, Processing Instruction, Text, Element, Entity reference, CDATA section

One thing to keep in mind is the parts of the XML document that are not exposed in this data model:

  • The XML declaration, including the version, standalone, and encoding declarations. These will be added as properties of the document node in DOM3, but they are not provided by current parsers.

  • Most information from the DTD and/or schema is not provided including element and attribute types and content models.

  • Any white space outside the root element.

  • Whether or not each character was provided by a character reference. Parsers may provide information about entity references, but are not required to do so.

A DOM program cannot manipulate any of these constructs. It cannot, for example, read in an XML document, then write it out again in the same encoding the original document used because it doesn’t know what encoding the original document used. It cannot treat $var differently than &#x24;var because it doesn’t know which was originally written.

Prev | Up | Next
Application Specific DOMs | Home | DOM Parsers for Java
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 08, 2002


DOM Parsers for Java
Prev Chapter 9. The Document Object Model Next

DOM Parsers for Java

DOM is defined almost completely in terms of interfaces rather than classes. Different parsers provide their own custom implementations of these standard interfaces. This offers a great deal of flexibility. Generally, you do not install the DOM interfaces on their own. Instead they come bundled with a parser distribution that provides the detailed implementation classes. DOM isn’t quite as broadly supported as SAX, but most of the major Java parsers provide it including Crimson, Xerces, XML for Java, the Oracle XML Parser for Java, and GNU JAXP.

DOM is not complete to itself. Almost all significant DOM programs need to use some parser-specific classes. DOM programs are not too difficult to port from one parser to another, but a recompile is normally required. You can’t just change a system property to switch from one parser to another as you can with SAX. In particular, DOM2 does not specify how one parses a document, creates a new document, or serializes a document into a file or onto a stream. These important functions are all performed by parser-specific classes.

JAXP, the Java API for XML Processing, fills in a few of the holes in DOM by providing standard parser independent means to parse existing documents, create new documents, and serialize in-memory DOM trees to XML files. Most current Java parsers that support DOM Level 2 also support JAXP 1.1. JAXP is a standard part of Java 1.4. JAXP is not included in earlier versions of Java, but it does work with Java 1.1 and later and is bundled with most parser class libraries.

DOM Level 3 also promises to fill the same holes JAXP fills (parsing, serializing, and bootstrapping). However, it is not yet finished and not yet supported in a large way by any parsers.

Because DOM depends so heavily on parser classes, its performance characteristics vary widely from one parser to the next. Speed is something of a concern, but memory consumption is a much bigger issue for most applications. All DOM implementations I’ve seen use more space for the in-memory DOM tree than the actual file on the disk occupies. Generally the in-memory DOM trees range from three to ten times as large as the actual XML text. Some parsers including Xerces offer a “lazy DOM” that leaves most of the document on the disk, and only reads into memory those parts of the document the client actually requests.

Measuring DOM Size

To test the memory usage of various implementations, I wrote a simple program that loaded the XML specification, 2nd edition, into a DOM Document object. The spec’s text format is 197K (not including the DTD which adds another 56K, but isn't really modeled by DOM at all). Here's the approximate amount of memory used by the Document objects built from this file by several parsers:

  • Xerces-J 2.0.1: 1489K

  • Crimson 1.1.3 (JDK 1.4 default): 1230K

  • Oracle XML Parser for Java 9.2.0.2.0: 2500K

I used a couple of different techniques to measure the memory used. In one case I used OptimizeIt and the Java Virtual Machine Profiling Interface (JVMPI) to check the heap size. I ran the program both with and without loading the document. I subtracted the total heap memory used without loading the document from the memory used when the document was loaded to get the numbers reported above. In the other test, I used the Runtime class to measure the total memory and the free memory before and after the Document was created. In both cases, I garbage collected before taking the final measurements. The results from the separate tests were within 15% of each other. All tests were performed in Sun’s JDK 1.4.0 using Hotspot on Windows NT 4.0SP6.

I don't claim these numbers are perfect, and I certainly don’t think this one test document justifies any claims whatsoever about the relative efficiency of the different DOM implementations. The difference between Crimson and Xerces is well within my margin of error. A more serious test would have to look at how the different implementations scale with the size of the initial document, and perhaps graph the curves of memory size vs. file size. For instance, it's possible that each of these requires a minimum of 1024K per document, but grows relatively slowly after that point. I did run the same tests on a minimal document that contained a single empty element. The results ranged from 3K to 131K for this document. However, these numbers were extremely sensitive to exactly when and how garbage was collected. I wouldn’t claim the results are accurate to better than ±300K. However, I do think that together these tests demonstrate just how inefficient DOM is.

Another distinguishing factor between different DOM implementations is the extra features the parser provides. Most parsers provide methods to parse XML documents and serialize DOM trees to XML. Other useful features include schema validation, database access, XInclude, XSLT, XPath, support for different character sets, and application specific DOMs like the MathML, SVG, and WML DOMs.

For example, the Oracle and Xerces parsers provide schema validation. Ælfred and Crimson don’t. Ælfred has partial support for XInclude. The other three don’t. The Oracle XML parser can produce a DOM Document object from a SQL query against a relational database or a JDBC ResultSet object. The other three can’t. The Oracle XML parser can decode the WAP binary XML format. The other three can’t. Xerces has specialized DOMs for HTML and WML documents. The other three don’t. These are all non-standard features; but if they’re useful to you, that would be a good reason to choose one parser over another. Table 9.2 summarizes parser support for various useful features.

Table 9.2. DOM Parser Features

 | Xerces | Ælfred | Oracle | Crimson
DTDs | X | X | X | X
Schemas | X |  | X |
Namespaces | X | X | X | X
Lazy DOM | X |  |  |
HTML DOM | X |  |  |
Views |  |  |  |
Style Sheets |  |  |  |
CSS |  |  |  |
CSS2 |  |  |  |
Events | X | X | X |
UI Events |  | X |  |
Mouse Events |  |  |  |
Mutation Events | X | X |  |
HTML Events |  | X |  |
Traversal | X | partial | X |
Range |  |  | X |
XSLT/XPath | Via Xalan-J |  | X |
XInclude |  | X |  |
Prev | Up | Next
Trees | Home | Parsing documents with a DOM Parser
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 27, 2002


Parsing documents with a DOM Parser
Prev Chapter 9. The Document Object Model Next

Parsing documents with a DOM Parser

Unlike SAX, DOM does not have a class or interface that represents the XML parser. Each parser vendor provides their own unique class. In Xerces, this is org.apache.xerces.parsers.DOMParser. In Crimson it’s org.apache.crimson.jaxp.DocumentBuilderImpl. In Ælfred it’s an inner class, gnu.xml.dom.JAXPFactory$JAXPBuilder. In Oracle, it’s oracle.xml.parser.v2.DOMParser In other implementations it will be something else.

Furthermore, since these classes do not share a common interface or superclass, the methods they use to parse documents vary too. For example, in Xerces, the two methods that read XML documents have these signatures:

public void parse(InputSource source)
    throws SAXException, IOException;
public void parse(String systemID)
    throws SAXException, IOException;

To get the Document object from the parser, you first call one of the parse methods and then call the getDocument() method.

public Document getDocument();

For example, if parser is a Xerces DOMParser object, then these lines of code load the DOM Core 2.0 specification into a DOM Document object named spec:

parser.parse("http://www.w3.org/TR/DOM-Level-2-Core");
Document spec = parser.getDocument();

In Crimson’s parser class, by contrast, the parse() method returns a Document object directly so no separate getDocument() method is needed. For example,

Document spec
 = parser.parse("http://www.w3.org/TR/DOM-Level-2-Core");

Furthermore, the Crimson parse() method is five-way overloaded instead of two:

public Document parse(InputSource source)
    throws SAXException, IOException;
public Document parse(String uri)
    throws SAXException, IOException;
public Document parse(File file)
    throws SAXException, IOException;
public Document parse(InputStream in)
    throws SAXException, IOException;
public Document parse(InputStream in, String systemID)
    throws SAXException, IOException;

Example 9.3 is a simple program that uses Xerces to check documents for well-formedness. You can see that it depends directly on the org.apache.xerces.parsers.DOMParser class.

Example 9.3. A program that uses Xerces to check documents for well-formedness

import org.apache.xerces.parsers.DOMParser;
import org.xml.sax.SAXException;
import java.io.IOException;

public class XercesChecker {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java XercesChecker URL");
      return;
    }
    String document = args[0];

    DOMParser parser = new DOMParser();
    try {
      parser.parse(document);
      System.out.println(document + " is well-formed.");
    }
    catch (SAXException e) {
      System.out.println(document + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }

  }

}

It’s not hard to port XercesChecker to a different parser like Oracle, but you do need to change the source code as shown in Example 9.4 and recompile.

Example 9.4. A program that uses the Oracle XML parser to check documents for well-formedness

import oracle.xml.parser.v2.*;
import org.xml.sax.SAXException;
import java.io.IOException;

public class OracleChecker {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java OracleChecker URL");
      return;
    }
    String document = args[0];

    DOMParser parser = new DOMParser();
    try {
      parser.parse(document);
      System.out.println(document + " is well-formed.");
    }
    catch (XMLParseException e) {
      System.out.println(document + " is not well-formed.");
      System.out.println(e);

    }
    catch (SAXException e) {
      System.out.println(document + " could not be parsed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }

  }

}

Other parsers have slightly different methods still. What all of these have in common is that they read an XML document from a source of text, most commonly a file or a stream, and provide an org.w3c.dom.Document object. Once you have a reference to this Document object you can work with it using only the standard methods of the DOM interfaces. There’s no further need to use parser-specific classes.

JAXP DocumentBuilder and DocumentBuilderFactory

The lack of a standard means of parsing an XML document is one of the holes that JAXP fills. If your parser implements JAXP, then instead of using the parser-specific classes, you can use the javax.xml.parsers.DocumentBuilderFactory and javax.xml.parsers.DocumentBuilder classes to parse the documents. The basic approach is as follows:

  • Use the static DocumentBuilderFactory.newInstance() factory method to return a DocumentBuilderFactory object.

  • Use the newDocumentBuilder() method of this DocumentBuilderFactory object to return a parser-specific instance of the abstract DocumentBuilder class.

  • Use one of the five parse() methods of DocumentBuilder to read the XML document and return an org.w3c.dom.Document object.

Example 9.5 demonstrates with a simple program that uses JAXP to check documents for well-formedness.

Example 9.5. A program that uses JAXP to check documents for well-formedness

import javax.xml.parsers.*; // JAXP
import org.xml.sax.SAXException;
import java.io.IOException;

public class JAXPChecker {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java JAXPChecker URL");
      return;
    }
    String document = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      DocumentBuilder parser = factory.newDocumentBuilder();
      parser.parse(document);
      System.out.println(document + " is well-formed.");
    }
    catch (SAXException e) {
      System.out.println(document + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }
    catch (FactoryConfigurationError e) {
      // JAXP suffers from excessive brain-damage caused by
      // intellectual in-breeding at Sun. (Basically the Sun
      // engineers spend way too much time talking to each other
      // and not nearly enough time talking to people outside
      // Sun.) Fortunately, you can happily ignore most of the
      // JAXP brain damage and not be any the poorer for it.

      // This, however, is one of the few problems you can't
      // avoid if you're going to use JAXP at all.
      // DocumentBuilderFactory.newInstance() should throw a
      // ClassNotFoundException if it can't locate the factory
      // class. However, what it does throw is an Error,
      // specifically a FactoryConfigurationError. Very few
      // programs are prepared to respond to errors as opposed
      // to exceptions. You should catch this error in your
      // JAXP programs as quickly as possible even though the
      // compiler won't require you to, and you should
      // never rethrow it or otherwise let it escape from the
      // method that produced it.
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  }

}

For example, here’s the output from when I ran this program across this chapter’s DocBook source code:

D:\books\XMLJAVA>java JAXPChecker file:///D:/books/xmljava/dom.xml
file:///D:/books/xmljava/dom.xml is well-formed.

How JAXP Chooses Parsers

You may be wondering which parser this program actually uses. JAXP, after all, is reasonably parser-independent. The answer depends on which parsers are installed in your class path and how certain system properties are set. The default is to use the class named by the javax.xml.parsers.DocumentBuilderFactory system property. For example, if you want to make sure that Xerces is used to parse documents, then you would run JAXPChecker like this:

D:\books\XMLJAVA>java
 -Djavax.xml.parsers.DocumentBuilderFactory=org.apache.xerces.jaxp.DocumentBuilderFactoryImpl
 JAXPChecker file:///D:/books/xmljava/dom.xml
file:///D:/books/xmljava/dom.xml is well-formed.

If the javax.xml.parsers.DocumentBuilderFactory property is not set, then JAXP looks in the lib/jaxp.properties properties file in the JRE directory to determine a default value for the javax.xml.parsers.DocumentBuilderFactory system property. If you want to consistently use a certain DOM parser, for instance gnu.xml.dom.JAXPFactory, place the following line in that file:

javax.xml.parsers.DocumentBuilderFactory=gnu.xml.dom.JAXPFactory

If this fails to locate a parser, next JAXP looks for a META-INF/services/javax.xml.parsers.DocumentBuilderFactory file in all JAR files available to the runtime to find the name of the concrete DocumentBuilderFactory subclass.

Finally, if that fails, then DocumentBuilderFactory.newInstance() returns a default class, generally the parser from the vendor who also provided the JAXP classes. For example, the JDK JAXP classes pick org.apache.crimson.jaxp.DocumentBuilderFactoryImpl by default but the Ælfred JAXP classes pick gnu.xml.dom.JAXPFactory instead.

Configuring DocumentBuilderFactory

The DocumentBuilderFactory has a number of options that allow you to determine exactly how the parsers it creates behave. Most of the setter methods take a boolean that turns the feature on if true or off if false. However, a couple of the features are defined as confusing double negatives, so read carefully.

Coalescing

These two methods determine whether CDATA sections are merged with text nodes or not. If the coalescing feature is true, then the result tree will not contain any CDATA section nodes, even if the parsed XML document does contain CDATA sections.

public boolean isCoalescing();
public void setCoalescing(boolean coalescing);

The default is false, but in most situations you should set this to true, especially if you’re just reading the document and are not going to write it back out again. CDATA sections should not be treated differently than any other text. Whether or not certain text is written in a CDATA section should be purely a matter of syntax sugar for human convenience, not anything that has any effect on the data model.

Expand Entity References

The following two methods determine whether the parsers produced by this factory expand entity references.

public boolean isExpandEntityReferences();
public void setExpandEntityReferences(boolean expandEntityReferences);

The default is true. If a parser is validating, then this it will expand entity references, even if this feature is set to false. That is, the validation feature overrides the expand entity references feature.

The five predefined references— &amp;, &lt;, &gt;, &quot;, and &apos; —will always be expanded regardless of the value of this property.

Ignore Comments

The following two methods determine whether the parsers produced by this factory will generate comment nodes for comments seen in the input document. The default, false, means that comment nodes will be produced. (Watch out for the double negative here. False means include comments, and true means don’t include comments. This confused me initially, and I was getting my poison pen all ready to write about the brain damage of throwing away comments although the spec required them to be included, when I realized that the method was in fact behaving like it should.)

public boolean isIgnoringComments();
public void setIgnoringComments(boolean ignoringComments);
Ignore Element Content Whitespace

The following two methods determine whether the parsers produced by this factory will generate text nodes for so-called “ignorable white space”; that is, white space that occurs between tags where the DTD specifies that parsed character data cannot appear.

public boolean isIgnoringElementContentWhitespace();
public void setIgnoringElementContentWhitespace(boolean ignoreElementContentWhitespace);

The default is false; that is, include text nodes for ignorable white space. Setting this to true might well be useful in record-like documents. However, for this property to make a difference, the documents must have a DTD and should be valid or very nearly so. Otherwise the parser can’t tell which white space is ignorable and which isn’t.

Namespace Aware

The following two methods determine whether the parsers produced by this factory are “namespace aware.” A namespace aware parser will set the prefix and namespace URI properties of element and attribute nodes that are in a namespace. A non-namespace aware parser won’t.

public boolean isNamespaceAware();
public void setNamespaceAware(boolean namespaceAware);

The default is false, which is truly brain-damaged. You should always set this to true. For example,

DocumentBuilderFactory factory
 = DocumentBuilderFactory.newInstance();
factory.setNamespaceAware(true);
Validating

These methods determine whether or not the parsers produced by this factory validate the document against its DTD.

public boolean isValidating();
public void setValidating(boolean validating);

The default is false, do not validate. If you want to validate your documents, set this property to true. You’ll also need to register a SAX ErrorHandler with the DocumentBuilder using its setErrorHandler() method to receive notice of validity errors. Example 9.6 demonstrates with a program that uses JAXP to validate a document named on the command line.

Example 9.6. Using JAXP to check documents for well-formedness

import javax.xml.parsers.*; // JAXP
import org.xml.sax.*;
import java.io.IOException;

public class JAXPValidator {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java JAXPValidator URL");
      return;
    }
    String document = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      // Always turn on namespace awareness
      factory.setNamespaceAware(true);
      // Turn on validation
      factory.setValidating(true);

      DocumentBuilder parser = factory.newDocumentBuilder();

      // SAXValidator was developed in Chapter 7
      ErrorHandler handler = new SAXValidator();
      parser.setErrorHandler(handler);

      parser.parse(document);
      if (handler.isValid()) {
        System.out.println(document + " is valid.");
      }
      else {
        // If the document isn't well-formed, an exception has
        // already been thrown and this has been skipped.
        System.out.println(document + " is well-formed.");
      }

    }
    catch (SAXException e) {
      System.out.println(document + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  }

}
Parser-specific Attributes

Many JAXP aware parsers support various custom features. For example, Xerces, has an http://apache.org/xml/features/dom/create-entity-ref-nodes feature that lets you choose whether or not to include entity reference nodes in the DOM tree. This is not the same as deciding whether or not to expand entity references. That determines whether the entity nodes that are placed in the tree have children representing their replacement text or not.

JAXP allows you to set and get these custom features as objects of the appropriate type using these two methods:

public Object getAttribute(String name)
    throws IllegalArgumentException;
public void setAttribute(String name, Object value)
    throws IllegalArgumentException;

For example, suppose you’re using Xerces and you don’t want to include entity reference nodes. They're included by default so you need to set http://apache.org/xml/features/dom/create-entity-ref-nodes to false. You would use setAttribute() on the DocumentBuilderFactory like this:

DocumentBuilderFactory factory
 = DocumentBuilderFactory.newInstance();
 factory.setAttribute(
  "http://apache.org/xml/features/dom/create-entity-ref-nodes",
  new Boolean(false)
 );

The naming conventions for both attribute names and values depends on the underlying parser. Xerces uses URL strings like SAX feature names. Other parsers may do something different. JAXP 1.2 will add a couple of standard attributes related to schema validation.

DOM3 Load and Save

JAXP only works for Java, and it is a Sun proprietary standard. Consequently, the W3C DOM working group is preparing an alternative cross-vendor means of parsing an XML document with a DOM parser. This will be published as part of DOM Level 3. DOM3 is not close to a finished recommendation at the time of this writing and is not yet implemented by any parsers, but I can show you pretty much what the interface is likely to look like.

Parsing a document with DOM3 requires four steps:

  1. Load a DOMImplementation object by passing the feature string "LS-Load 3.0" to the DOMImplementationRegistry.getDOMImplementation() factory method. (This class is also new in DOM3.)

  2. Cast this DOMImplementation object to DOMImplementationLS, the sub-interface that provides the extra methods you need.

  3. Call the implementation’s createDOMBuilder() method to create a new DOMBuilder object. This is the new DOM3 class that represents the parser. The first argument to createDOMBuilder() specifies whether the document is parsed synchronously or asynchronously. The second argument is a URL identifying the type of schema to be used during the parse, "http://www.w3.org/2001/XMLSchema" for W3C XML Schemas, "http://www.w3.org/TR/REC-xml" for DTDs. You can pass null to ignore all schemas.

  4. Pass the document’s URL to the the builder object’s parseURI() method to read the document and return a Document object.

Example 9.7 demonstrates with a simple program that uses DOM3 to check documents for well-formedness.

Example 9.7. A program that uses DOM3 to check documents for well-formedness

import org.w3c.dom.*;
import org.w3c.dom.ls.*;

public class DOM3Checker {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java DOM3Checker URL");
      return;
    }
    String document = args[0];

    try {
      DOMImplementationLS impl = (DOMImplementationLS)
       DOMImplementationRegistry
       .getDOMImplementation("LS-Load 3.0");
      DOMBuilder parser = impl.createDOMBuilder(
       DOMImplementationLS.MODE_SYNCHRONOUS,
       "http://www.w3.org/TR/REC-xml");
    // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    // Use DTDs when parsing
      Document doc = parser.parseURI(document);
      System.out.println(document + " is well-formed.");
    }
    catch (NullPointerException e) {
      System.err.println("The current DOM implementation does"
       + " not support DOM Level 3 Load and Save");
    }
    catch (DOMException e) {
      System.err.println(document + " is not well-formed");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check "
       + document
      );
    }
    catch (Exception e) {
      // Probably a ClassNotFoundException,
      // InstantiationException, or IllegalAccessException
      // thrown by DOMImplementationRegistry.getDOMImplementation
      System.out.println("Probable CLASSPATH problem.");
      e.printStackTrace();
    }

  }

}

For the time being, JAXP’s DocumentBuilderFactory is the obvious choice since it works today and is supported by almost all DOM parsers written in Java. Longer term, DOM3 will provide a number of important capabilities JAXP does not, including parse progress notification and document filtering. However, since these APIs are far from ready for prime time just yet, for the rest of this book, I’m mostly going to use JAXP without further comment.

Prev | Up | Next
DOM Parsers for Java | Home | The Node Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


The Node Interface
Prev Chapter 9. The Document Object Model Next

The Node Interface

Once you’ve parsed the document and formed an org.w3c.dom.Document object, you can forget about the differences between the various parsers and just work with the standard DOM interfaces (at least until you want to write the document back out to a file again. Then you have to consider parser-specific classes or JAXP once more.)

All the nodes in the tree are represented by instances of the Node interface summarized in Example 9.8.

Example 9.8. The Node interface

package org.w3c.dom;

public interface Node {

  // Node type constants
  public static final short ELEMENT_NODE                = 1;
  public static final short ATTRIBUTE_NODE              = 2;
  public static final short TEXT_NODE                   = 3;
  public static final short CDATA_SECTION_NODE          = 4;
  public static final short ENTITY_REFERENCE_NODE       = 5;
  public static final short ENTITY_NODE                 = 6;
  public static final short PROCESSING_INSTRUCTION_NODE = 7;
  public static final short COMMENT_NODE                = 8;
  public static final short DOCUMENT_NODE               = 9;
  public static final short DOCUMENT_TYPE_NODE          = 10;
  public static final short DOCUMENT_FRAGMENT_NODE      = 11;
  public static final short NOTATION_NODE               = 12;

  // Node properties
  public String   getNodeName();
  public String   getNodeValue() throws DOMException;
  public void     setNodeValue(String nodeValue)
   throws DOMException;
  public short    getNodeType();
  public String   getNamespaceURI();
  public String   getPrefix();
  public void     setPrefix(String prefix) throws DOMException;
  public String   getLocalName();

  // Navigation methods
  public Node         getParentNode();
  public boolean      hasChildNodes();
  public NodeList     getChildNodes();
  public Node         getFirstChild();
  public Node         getLastChild();
  public Node         getPreviousSibling();
  public Node         getNextSibling();
  public Document     getOwnerDocument();
  public boolean      hasAttributes();
  public NamedNodeMap getAttributes();

  // Manipulator methods
  public Node insertBefore(Node newChild, Node refChild)
   throws DOMException;
  public Node replaceChild(Node newChild,  Node oldChild)
   throws DOMException;
  public Node removeChild(Node oldChild) throws DOMException;
  public Node appendChild(Node newChild) throws DOMException;

  // Utility methods
  public Node cloneNode(boolean deep);
  public void normalize();
  public boolean isSupported(String feature, String version);

}

You can do quite a lot with just this interface alone. You can add, move, remove, and copy nodes in the tree. You can walk the tree while reading the names and values of everything in the tree. This interface can be roughly divided into five sections: node type constants, methods to set and get node properties, methods to navigate the DOM tree, methods to add and remove children of a node, and a few utility methods. Let’s take them in that order.

Node Types

There are twelve constants for the twelve named node types defined in the DOM core, and a method that returns the type of the current node using one of these constants. To a Java programmer, these are just weird all around. First of all, you’d probably expect to use instanceof, getClass(), and class names to test for types when necessary instead of short constants and a getNodeType() method. And even if for some strange reason you did use named constants, you’d probably use the type-safe enum pattern if you were familiar with it or ints if you weren’t. Either way, a short constant is just plain weird.

What’s going on here is that DOM is not designed in or for Java. It is written in IDL and intended for all object-oriented languages, including C++, Python, Perl, JavaScript, and more. This means it has to make a lot of compromises to support the broad range of capabilities of those different languages. For example, AppleScript doesn’t have any equivalent to Java’s instanceof operator that allows it to test whether a variable is an instance of a particular class. Prior to version 1.4, JavaScript didn’t have one either. Some older C++ compilers don’t support runtime type information. (RTTI) and no C compilers do. Consequently, the DOM can’t rely on these features because it has to work in those languages. Therefore, it has to reinvent things Java already has.

Note

Using a getNodeType() method also allows a single class to implements more than one of the standard interfaces, which is possible because Java supports multiple interface inheritance. For instance, an implementation might use a single NodeImpl class for all twelve different sub-interfaces of Node. Then, an object could simultaneously be an instance of Comment, Element, Text, and all the other things besides. I've seen exactly one DOM implementation that does this. The Saxon XSLT processor (about which more in Chapter 16) uses its NodeImpl to represent all non-document and and non-element nodes. However, all the general purpose DOM implementations I've encountered use a separate class for each separate node type.

The issue of the short constants is a little different. Here, DOM has just chosen to implement a different language’s idioms rather than Java’s. In this case, it’s following the C++ conventions, where shorts and short constants are much more common than they are in Java. As for using integers instead of type-safe enums, I suspect the DOM group simply felt that type-safe enums were too complicated to implement in IDL (if they considered the possibility of type-safe enums at all). After all, this whole set of node types is really just a hack for languages whose reflection isn’t as complete as Java’s.

Example 9.9 is a simple utility class that uses the getNodeType() method and these constants to returns a string specifying the type of the node. In itself, it isn’t very interesting; but I’ll need this for a few of the later programs.

Example 9.9. Changing short type constants to strings

import org.w3c.dom.Node;

public class NodeTyper {

   public static String getTypeName(Node node) {

    int type = node.getNodeType();
    /* Yes, getNodeType() returns a short, but Java will
       almost always upcast this short to an int before
       using it in any operation, so we might as well just go
       ahead and use the int in the first place. */

    switch (type) {
      case Node.ELEMENT_NODE: return "Element";
      case Node.ATTRIBUTE_NODE: return "Attribute";
      case Node.TEXT_NODE: return "Text";
      case Node.CDATA_SECTION_NODE: return "CDATA Section";
      case Node.ENTITY_REFERENCE_NODE: return "Entity Reference";
      case Node.ENTITY_NODE: return "Entity";
      case Node.PROCESSING_INSTRUCTION_NODE: return "Processing Instruction";
      case Node.COMMENT_NODE: return "Comment";
      case Node.DOCUMENT_NODE: return "Document";
      case Node.DOCUMENT_TYPE_NODE: return "Document Type Declaration";
      case Node.DOCUMENT_FRAGMENT_NODE: return "Document Fragment";
      case Node.NOTATION_NODE: return "Notation";
      default: return "Unknown Type";
   /* It is possible for the default case to be
      reached. DOM only defines 12 kinds of nodes, but other
      application specific DOMs can add their own as well.
      You're not likely to encounter these while parsing an
      XML document with a standard parser, but you might
      encounter such things with custom parsers designed for
      non-XML documents. DOM Level 3 XPath does define a
      thirteenth kind of node, XPathNamespace. */
    }

  }

}

Node Properties

The next batch of methods allow you to get and, in a couple of cases, set the common node properties. Although all nodes have these methods, they don’t necessarily return a sensible value for every kind of node. For example, only element and attribute nodes have namespace URIs. getNamespaceURI() returns null when invoked on any other kind of node. The getNodeName() method returns the complete name for nodes that have names, and #node-type for nodes that don’t have names; i.e. #document, #text, #comment, etc.

public String getNodeName();
public String getNodeValue()
    throws DOMException;
public String setNodeValue(String value)
    throws DOMException;
public short getNodeType();
public String getNamespaceURI();
public String getPrefix();
public void setPrefix(String prefix)
    throws DOMException;
public String getLocalName();

Example 9.10 is another simple utility class that accepts a Node as an argument and prints out the values of its non-null properties. Again, I’ll be using this class shortly in another program.

Example 9.10. A class to inspect the properties of a node

import org.w3c.dom.*;
import java.io.*;

public class PropertyPrinter {

  private Writer out;

  public PropertyPrinter(Writer out) {
    if (out == null) {
      throw new NullPointerException("Writer must be non-null.");
    }
    this.out = out;
  }

  public PropertyPrinter() {
    this(new OutputStreamWriter(System.out));
  }

  private int nodeCount = 0;

  public void writeNode(Node node) throws IOException {

    if (node == null) {
      throw new NullPointerException("Node must be non-null.");
    }
    if (node.getNodeType() == Node.DOCUMENT_NODE
     || node.getNodeType() == Node.DOCUMENT_FRAGMENT_NODE) {
      // starting a new document, reset the node count
      nodeCount = 1;
    }

    String name      = node.getNodeName(); // never null
    String type      = NodeTyper.getTypeName(node); // never null
    String localName = node.getLocalName();
    String uri       = node.getNamespaceURI();
    String prefix    = node.getPrefix();
    String value     = node.getNodeValue();

    StringBuffer result = new StringBuffer();
    result.append("Node " + nodeCount + ":\r\n");
    result.append("  Type: " + type + "\r\n");
    result.append("  Name: " + name + "\r\n");
    if (localName != null) {
      result.append("  Local Name: " + localName + "\r\n");
    }
    if (prefix != null) {
      result.append("  Prefix: " + prefix + "\r\n");
    }
    if (uri != null) {
      result.append("  Namespace URI: " + uri + "\r\n");
    }
    if (value != null) {
      result.append("  Value: " + value + "\r\n");
    }

    out.write(result.toString());
    out.write("\r\n");
    out.flush();

    nodeCount++;

  }

}

Notice once again the use of polymorphism. The writeNode() method operates on a Node object without any clue what its actual type is. It prints the properties of the node onto the configured Writer in the following form:

Node 16:
  Type: Text
  Name: #text
  Value: RHAT

The format changes depending on what kind of node is passed to it.

There are also two methods in the Node interface that can change a node. setPrefix() changes a node’s namespace prefix. Trying to use an illegal or reserved prefix throws a DOMException. This method has no effect on anything except an element or an attribute node.

The setValue() method changes the node’s string value. It can be used on comment, text, processing instruction, and CDATA section nodes. It has no effect on other kinds of nodes. It throws a DOMException if the node you’re setting is read-only (as a text node might be inside an entity node.)

The remaining properties cannot be set from the Node interface. To change names, URIs, and such you have to use the more specific interfaces like Element and Attr. Most of the time you’re better off using the more detailed sub-interfaces if you’re trying to change a tree, anyway.

Navigating the tree

The third batch of methods allow you to navigate the tree by finding the parent, first child, last child, previous and next siblings, and attributes of any node. Since not all nodes have children, you should test for the presence of these things with hasChildren() before calling the getFirstChild() and getLastChild() methods. You should also be prepared for any of these methods to return null in the event that the requested node doesn’t exist. Similarly, you should check hasAttributes() before calling the getAttributes() method.

Example 9.11 demonstrates with a simple program that recursively traverses the tree in a preorder fashion. As each node is visited, its name and value is printed using last section’s PropertyPrinter class. Once again, Node is the only class used from DOM. That’s the power of polymorphism. You can do quite a lot without knowing exactly what it is you’re doing it to.

Example 9.11. Walking the tree with the Node interface

import javax.xml.parsers.*;  // JAXP
import org.w3c.dom.Node;
import org.xml.sax.SAXException;
import java.io.IOException;

public class TreeReporter {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java TreeReporter URL");
      return;
    }

    TreeReporter iterator = new TreeReporter();
    try {
      // Use JAXP to find a parser
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      // Turn on namespace support
      factory.setNamespaceAware(true);
      DocumentBuilder parser = factory.newDocumentBuilder();

      // Read the entire document into memory
      Node document = parser.parse(args[0]);

      // Process it starting at the root
      iterator.followNode(document);

    }
    catch (SAXException e) {
      System.out.println(args[0] + " is not well-formed.");
      System.out.println(e.getMessage());
    }
    catch (IOException e) {
      System.out.println(e);
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  } // end main

  private PropertyPrinter printer = new PropertyPrinter();

  // note use of recursion
  public void followNode(Node node) throws IOException {

    printer.writeNode(node);
    if (node.hasChildNodes()) {
      Node firstChild = node.getFirstChild();
      followNode(firstChild);
    }
    Node nextNode = node.getNextSibling();
    if (nextNode != null) followNode(nextNode);

  }

}

Here’s the beginning of the output produced by running this program across Example 9.2:

% java TreeReporter getQuote.xml
Node 1:
  Type: Document
  Name: #document

Node 2:
  Type: Processing Instruction
  Name: xml-stylesheet
  Value: type="text/css" href="xml-rpc.css"

Node 3:
  Type: Comment
  Name: #comment
  Value:  It’s unusual to have an xml-stylesheet processing
     instruction in an XML-RPC document but it is legal, unlike
     SOAP where processing instructions are forbidden.

Node 4:
  Type: Document Type Declaration
  Name: methodCall

Node 5:
  Type: Element
  Name: methodCall
  …

The key to this program is the followNode() method. It first writes the node using the PropertyPrinter, then recursively invokes followNode() on the current node’s first child and then its next sibling. This is equivalent to XPath document order (children come before siblings). The hasChildNodes() method tests whether there actually are children before asking for the first child node. For siblings, we have to retrieve the next sibling whether there is one or not, and then check to see whether it’s null before dereferencing it.

TreeReporter is actually very raw. As you’ll see, DOM provides a lot of helper classes that make operations like this much simpler to code. However, it never hurts to keep in mind what all those helper classes are doing behind the scenes, which is in fact something very much like this.

Modifying the tree

The Node interface has four methods that change the tree by inserting, removing, replacing, and appending children at points specified by nodes in the tree:

public Node insertBefore(Node toBeInserted, Node toBeInsertedBefore)
    throws DOMException;
public Node replaceChild(Node toBeInserted, Node toBeReplaced)
    throws DOMException;
public Node removeChild(Node toBeRemoved)
    throws DOMException;
public Node appendChild(Node toBeAppended)
    throws DOMException;

All four of these methods throw a DOMException if you try to use them to make a document malformed; for instance, by removing the root element or appending a child to a text node. All four methods return the node being inserted/replaced/removed/appended.

You can only use these methods to move nodes around in the same document. Although removeChild() and replaceChild() disconnect nodes from a document’s tree, they do not change those nodes’ owner document. The disconnected nodes cannot be placed in a different document. Nodes are only allowed to be placed in the document where they begin their life. Moving a node from one document to another requires importing it, a technique which I’ll take up in the next chapter.

It’s hard to come up with a plausible example of these methods until I’ve shown you how to create new nodes. That will be coming in the next chapter. In the meantime, Example 9.12 is a program that moves all processing instruction nodes from inside the root element to before the root element and all comment nodes from inside the root element to after the root element. For example, this document:

<?xml version="1.0"?>
<document>
  Some data
  <!-- first comment -->
  <?example first processing instruction ?>
  Some more data
  <!-- second comment -->
  <?example second processing instruction ?>
  <empty/>
</document>

would become this document:

<?xml version="1.0" encoding="utf-8"?>
<?example first processing instruction ?>
<?example second processing instruction ?><document>
  Some data

  Some more data

  <empty/>
</document><!-- first comment --><!-- second comment -->

I don’t actually think this is a sensible thing to do. In particular, it inaccurately implies that comments and processing instructions can be removed and reordered willy-nilly without changing anything significant, which is not true in general. This is just the best example of these methods I could come up with without using too many classes and interfaces we haven’t covered yet.

Example 9.12. A method that changes a document by reordering nodes

import org.w3c.dom.*;

public class Restructurer {

  // Since this method only operates on its argument and does
  // not interact with any fields in the class, it's
  // plausibly made static.
  public static void processNode(Node current)
   throws DOMException {

    // I need to store a reference to the current node's next
    // sibling before we delete the node from the tree, in which
    // case it no longer has a sibling
    Node nextSibling = current.getNextSibling();

    int nodeType = current.getNodeType();
    if (nodeType == Node.COMMENT_NODE
     || nodeType == Node.PROCESSING_INSTRUCTION_NODE) {

      Node document = current.getOwnerDocument();
      // Find the root element by looping through the children of
      // the document until we find the only one that's an
      // element node. There's a quicker way to do this once we
      // learn more about the Document class in the next chapter.
      Node root = document.getFirstChild();
      while (!(root.getNodeType() == Node.ELEMENT_NODE )) {
        root = root.getNextSibling();
      }

      Node parent = current.getParentNode();
      parent.removeChild(current);
      if (nodeType == Node.COMMENT_NODE) {
        document.appendChild(current);
      }
      else if (nodeType == Node.PROCESSING_INSTRUCTION_NODE) {
        document.insertBefore(current, root);
      }

    }
    else if (current.hasChildNodes()) {
      Node firstChild = current.getFirstChild();
      processNode(firstChild);
    }

    if (nextSibling != null) {
      processNode(nextSibling);
    }

  }

}

This program walks the tree, calling the removeChild() method every time a comment or processing instruction node is spotted, and then inserting the processing instruction nodes before the root element with insertBefore() and the comment nodes after the root element with appendChild(). Both references to the document node, the root element node, and the nearest parent element node have to be stored at all times. The Document object is modified in place.

This program does not provide any means of outputting the changed document into a file where you can look at it. That too is coming.

Utility Methods

Finally, there are three assorted utility methods:

public Node cloneNode(boolean deep);
public void normalize();
public void isSupported(String feature, String version);

normalize()

The normalize() method descends the tree from the given node and merges all adjacent text nodes. Empty text nodes are deleted. This operation makes DOM roughly equivalent to an XPath data model in which each text node contains the maximum contiguous run of text not interrupted by markup. However, normalize() does not merge CDATA section nodes, which XPath would require.

The easiest approach is just to invoke normalize() on the Document object as soon as you get it. For example,

  Document document = parser.parse(document);
  document.normalize();

cloneNode()

The cloneNode() method makes a copy of the given node. If the deep argument is true, then the copy contains the full contents of the node including all its descendants. If the deep argument is false, then the clone does not contain copies of the original node’s children. The cloned node is disconnected; that is, it is not a child of the original node’s parent. However, it does belong to the original node’s document, even though it doesn’t have a position in that document’s tree. It can be added via insertBefore() or appendNode(), or replaceNode(). Conversely, the clone cannot be inserted into a different document. To make a copy for a different document, use the importNode() method in the Document interface instead. We’ll look at this in the next chapter.

isSupported()

The isSupported() method determines whether or not this node provides a given feature. For instance, you can pass the string "Events" to this method to find out whether or not this one node supports the events module. The version number for all DOM2 features is 2.0.

The isSupported() method isn’t used much, since there’s not a lot of point to asking for the features an individual node supports. There’s a similar method named hasFeature() in the DOMImplementation interface that’s more useful.

Prev | Up | Next
Parsing documents with a DOM Parser | Home | The NodeList interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified March 06, 2002


The NodeList interface
Prev Chapter 9. The Document Object Model Next

The NodeList interface

DOM stores the lists of children of each node in NodeList objects. This is a very basic indexed list shown in Example 9.13. Indexes start from 0 and continue to one less than length of the list, just like Java arrays.

Example 9.13. The NodeList interface

package org.w3c.dom;

public interface NodeList {

  public Node item(int index);
  public int  getLength();

}

Instances of this interface are returned by the getChildNodes() method in the Node interface, as well by various methods in its sub-interfaces we’ll encounter in the next chapter.

The actual data structure that backs this interface can be a linked list, an array, or something else. Details vary from implementation to implementation. Whatever the concrete data structure is, you can use node lists to simplify operations that iterate over children. For example, the processNode() method in Example 9.11 could be rewritten using NodeList instead of getNextSibling() like this:

  public void followNode(Node node) throws IOException {

    printer.writeNode(node);

    // Process the children
    NodeList children = node.getChildNodes();
    for (int i = 0; i < children.getLength(); i++) {
      Node child = children.item(i);
      followNode(child); // recursion
    }

  }

This still walks the tree like several of the earlier programs. However, the algorithm is somewhat more obvious because it uses more list iteration and less recursion. (Recursion is still necessary to descend the tree but not to move from one sibling to the next.)

Whether or not this variant is more efficient than the original version that only uses the Node interface depends on the concrete implementation. It may indeed be faster if the implementation classes store children in arrays. It may not be if the implementation classes use linked lists. Either way the difference is not likely to be significant. Which approach feels more natural to you is a lot more important than the marginal speed you might gain by picking one over the other.

Node lists are live. That is, if you add or delete a node from the list the change is reflected in the document and vice versa. This can make it a little tricky to keep track of where you are in the list since the length can keep changing, and nodes can move from one place in the list to another.

Node lists (and pretty much everything else in DOM) are not thread safe. If one thread is writing to or modifying a NodeList, while another thread is reading from it, data corruption is almost guaranteed. Because node lists are live, code can be unsafe even when no other thread has a reference to that particular node list, as long as some other thread has a reference to the Document from which the NodeList was built.

Prev | Up | Next
The Node Interface | Home | JAXP Serialization
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


JAXP Serialization
Prev Chapter 9. The Document Object Model Next

JAXP Serialization

Although DOM is a read-write API in memory, it’s sorely lacking when it comes to moving its in-memory data structure back out onto a disk, a network socket, or some other stream. Eventually, this omission will be rectified in DOM Level 3. In the meantime, you have the choice of either using implementation-specific serialization classes or using JAXP. The implementation-specific serialization classes generally provide more customization and features, but JAXP is enough for basic uses.

JAXP doesn’t include a serialization package, but you can hack basic output through the javax.xml.transform package by conveniently “forgetting” to install a transform. :-) The pattern is the same as parsing a document with JAXP. The basic steps are as follows:

  1. Use the static TransformerFactory.newInstance() factory method to return a javax.xml.transform.TransformerFactory object.

  2. Use the newTransformer() method of this TransformerFactory object to return an implementation-specific instance of the abstract javax.xml.transform.Transformer class.

  3. Construct a new javax.xml.transform.dom.DOMSource object from your DOM Document object.

  4. Construct a new javax.xml.transform.stream.StreamResult object connected to the OutputStream you want to write the document onto.

  5. Pass both the source and the result objects to the transform() method of the Transformer object created in step 2.

We can use this procedure to write a simple driver program for Example 9.12. Example 9.14 first uses JAXP to build a DOM Document object from a URL, then passes this object to the Restructurer.processNode() method, and finally serializes the whole document onto System.out.

Example 9.14. Using JAXP to both read and write an XML document

import javax.xml.parsers.*; // JAXP
import javax.xml.transform.*; // JAXP
import javax.xml.transform.dom.DOMSource; // JAXP
import javax.xml.transform.stream.StreamResult; // JAXP
import org.xml.sax.SAXException;
import org.w3c.dom.Document;
import java.io.IOException;

public class RestructureDriver {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java RestructureDriver URL");
      return;
    }
    String url = args[0];

    try {
      // Find a parser
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setNamespaceAware(true);
      DocumentBuilder parser = factory.newDocumentBuilder();

      // Read the document
      Document document = parser.parse(url);

      // Modify the document
      Restructurer.processNode(document);

      // Write it out again
      TransformerFactory xformFactory
       = TransformerFactory.newInstance();
      Transformer idTransform = xformFactory.newTransformer();
      Source input = new DOMSource(document);
      Result output = new StreamResult(System.out);
      idTransform.transform(input, output);

    }
    catch (SAXException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not read " + url
      );
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }
    catch (TransformerConfigurationException e) {
      System.out.println("This DOM does not support transforms.");
    }
    catch (TransformerException e) {
      System.out.println("Transform failed.");
    }

  }

}

You’ll learn how to actually use these classes for their intended purposes of XSLT transforms in the final chapter.

Prev | Up | Next
The NodeList interface | Home | DOMException
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


DOMException
Prev Chapter 9. The Document Object Model Next

DOMException

You’ve probably noticed that a lot of the methods so far have been declared to throw DOMException. This class itself is shown in Example 9.15 This is the generic exception for essentially anything that can go wrong while working with DOM ranging from logical errors like making an element one of its own children to implementation bugs. It is a runtime exception that does not have to be caught. However, you should always catch it or declare that your method throws it nonetheless. Conceptually, this should be a checked exception. However, many languages DOM supports, including C++ and Python, don’t have checked exceptions, so DOM uses runtime exceptions in order to keep the semantics of the various methods as similar as possible across languages.

Example 9.15. The DOMException class

package org.w3c.dom;

public class DOMException extends RuntimeException {

  public DOMException(short code, String message);

  public short code;

  public static final short INDEX_SIZE_ERR              = 1;
  public static final short DOMSTRING_SIZE_ERR          = 2;
  public static final short HIERARCHY_REQUEST_ERR       = 3;
  public static final short WRONG_DOCUMENT_ERR          = 4;
  public static final short INVALID_CHARACTER_ERR       = 5;
  public static final short NO_DATA_ALLOWED_ERR         = 6;
  public static final short NO_MODIFICATION_ALLOWED_ERR = 7;
  public static final short NOT_FOUND_ERR               = 8;
  public static final short NOT_SUPPORTED_ERR           = 9;
  public static final short INUSE_ATTRIBUTE_ERR         = 10;
  public static final short INVALID_STATE_ERR           = 11;
  public static final short SYNTAX_ERR                  = 12;
  public static final short INVALID_MODIFICATION_ERR    = 13;
  public static final short NAMESPACE_ERR               = 14;
  public static final short INVALID_ACCESS_ERR          = 15;

}

DOMException is the only exception that DOM standard methods throw. DOM methods don’t throw IOException, IllegalArgumentException, SAXException, or any other exceptions you may be familiar with from Java. In a few cases the implementation classes may throw a different exception, especially NullPointerException, and methods in non-DOM support classes like org.apache.xerces.parser.DOMParser or javax.xml.transform.dom.DOMResult can most certainly throw these exceptions. However, the DOM methods themselves don’t throw them.

Not only do DOM methods only throw DOMExceptions. They don’t even throw any subclasses of DOMException. Here, DOM is following C++ conventions rather than Java’s. Where Java tends to differentiate related exceptions through many different subclasses (In Java 1.4, there are more than fifty different subclasses of IOException alone.) DOM uses named short constants to identify the different problems that can arise. This is also useful for language like AppleScript where exceptions aren’t even classes. The exception code is exposed through DOMException’s public code field. The codes have the following meanings:

DOMException.DOMSTRING_SIZE_ERR

Something tried to put more than two billion characters into one string. This one isn’t too likely in Java. (If you’re trying to stuff that much text into one string, you’re going to have other problems long before DOM complains.) This exception is really meant for other languages with much smaller maximum string sizes.

DOMException.HIERARCHY_REQUEST_ERR

An attempt was made to add a node where it can’t go; e.g. making an element a child of a text node, an attribute a child of an element, adding a second root element to a document, or trying to make an element its own grandpa.

DOMException.INDEX_SIZE_ERR

A rare exception thrown by the splitText() method of a Text object if you try split the text before the beginning or after the end of the node.

DOMException.INUSE_ATTRIBUTE_ERR

You tried to add an existing Attr to a new element without removing it from the old element first.

DOMException.INVALID_ACCESS_ERR

The class that implements the DOM interface does not support the requested method, even though you’d normally expect it to.

DOMException.INVALID_CHARACTER_ERR

A Unicode character was used where it isn’t allowed; for example, an element name contained a dollar sign or a text node value contained a form feed. Many DOM implementations miss at least some problems that can occur with invalid characters. This exception is not thrown as often as it should be.

DOMException.INVALID_MODIFICATION_ERR

The class that implements the DOM interface cannot change the object in the requested way, even though you’d normally expect it to; e.g. it ran out of space for more child nodes.

DOMException.INVALID_STATE_ERR

The implementation class that backs the DOM interface you’re using has gotten confused, and cannot perform the requested operation. This would generally indicate a bug in the implementation.

DOMException.NAMESPACE_ERR

The namespace prefixes or URIs specified are somehow incorrect; e.g. the qualified name contains multiple colons, or the qualified name has a prefix but the namespace URI is null, or the prefix xml but the namespace URI is not http://www.w3.org/XML/1998/namespace.

DOMException.NOT_FOUND_ERR

A referenced node is not present in the document; e.g. you tried to remove an attribute the element does not have or tried to insert a node before another node that is no longer in the document.

DOMException.NOT_SUPPORTED_ERR

The implementation does not support the requested object type. For example, you tried to create a CDATA section node using an HTML document implementation.

DOMException.NO_DATA_ALLOWED_ERR

You tried to set the value of an element, document, document fragment, document type, entity, entity reference, or notation node. These kinds of nodes always have null values.

DOMException.NO_MODIFICATION_ALLOWED_ERR

You tried to change a read-only node. The most common reason a node is read-only is because it’s a descendant of an entity reference node.

DOMException.SYNTAX_ERR

You tried to set a value to a string that’s illegal in context; e.g. a comment value that contains the double hyphen -- or a CDATA section that contains the CDATA section end delimiter ]]>. In practice, most implementations do not watch for these sorts of syntax errors and do not throw this exception.

DOMException.WRONG_DOCUMENT_ERR

You tried to insert or add a node into a document other than its owner (the document that originally created the node). DOM does not allow nodes to change documents. Instead you have to use the importNode() method in the new Document to make a copy of the node you want to move.

Although there’s no way DOM can prevent programs from using error codes other than those listed here, the W3C has reserved all possible error codes for their own use. If you need something not listed here, write your own exception class or subclass DOMException. (Just because DOM doesn’t make full use of an object-oriented exception mechanism for reasons of compatibility with less object-oriented languages than Java doesn’t mean you shouldn’t do this in your pure Java code.)

Prev | Up | Next
JAXP Serialization | Home | Choosing between SAX and DOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 20, 2002


Choosing between SAX and DOM
Prev Chapter 9. The Document Object Model Next

Choosing between SAX and DOM

The single biggest factor in deciding whether to code your programs with SAX or DOM is programmer preference. SAX and DOM are very different APIs, Where SAX models the parser, DOM models the XML document. Most programmers find the DOM approach more to their taste, at least initially. Its pull model (The client program extracts the information it wants from a document by invoking various methods on that document.) is much more familiar than SAX’s push model (The parser tells you what it reads when it reads it, whether you’re ready for that information or not.)

However, SAX’s push model, unfamiliar as it is, can be much more efficient. SAX programs can be much faster than their DOM equivalents, and almost always use far less memory. In particular, SAX works extremely well when documents are streamed, and the individual parts of each document can be processed in isolation from other parts. If complicated processes can be broken down into serial filters, then SAX is hard-to-beat. SAX lends itself to assembly-line like automation where different stations perform small operations on just the parts of the document they have at hand right at that moment. By contrast, DOM is more like a factory where each worker operates only on an entire car. Every time the worker receives a new car off the line, they have to take the entire car apart to find the piece they need to work with, do their job, then put the car back together again before moving it along to the next worker. This system is not very efficient if there’s more than one station. DOM lends itself to monolithic applications where one program does everything. SAX works better when the program can be divided into small bits of independent work.

In particular the following characteristics indicate that a program should probably be using a streaming API such as SAX, XNI, or XMLPULL:

  • Documents will not fit into available memory. This is the only rule that really mandates one or the other. If your documents are too big for available memory, then you must use a streaming API such as SAX, painful though it may be. You really have no other choice.

  • You can process the document in small contiguous chunks of input. The entire document does not need to be available before you can do useful work.

    A slightly weaker variant of this is if the decisions you make only depend on preceding parts of the document, never on what comes later.

  • Processing can be divided up into a chain of successive operations.

However, if the problem matches this next set of characteristics, the program should probably be using DOM or perhaps another of the tree-based APIs such as JDOM:

  • The program needs to access widely separated parts of the document at the same time. Even more so, it needs access to multiple documents at the same time.

  • The internal data structures are almost as complicated as the document itself.

  • The program must modify the document repeatedly.

  • The program must store the document for a significant amount of time through many method calls, not just process it once and forget it.

On occasion, it’s possible to use both SAX and DOM. In particular, you can parse the document using a SAX XMLReader attached to a series of SAX filters, then use the final output from that process to construct a DOM Document. Working in reverse, you can traverse a DOM tree while firing off SAX events to a SAX ContentHandler.

The approach is the same Example 9.14 used earlier in to serialize a DOM Document onto a stream. Use JAXP to perform an identity transform from a source to a result. JAXP supports both SAX, DOM, and streams as sources and results. For example, this code fragment reads an XML document from the InputStream in and parses it with the SAX XMLReader named saxParser. Then it transforms this input into the equivalent DOMResult from which the DOM Document is extracted.

XMLReader saxParser = XMLReaderFactory.createXMLReader();
Source input = new SAXSource(saxParser, in);
Result output = new DOMResult();
TransformerFactory xformFactory
 = TransformerFactory.newInstance();
Transformer idTransform = xformFactory.newTransformer();
idTransform.transform(input, output);
Node document = idTransform.getNode();

To go in the other direction, from DOM to SAX, just use a DOMSource and a SAXResult. The DOMSource is constructed from a DOM Document object, and the SAXResult is configured with a ContentHandler:

Source input = new DOMSource(document);
ContentHandler handler = new MyContentHandler();
Result output = new SAXResult(handler);
TransformerFactory xformFactory
 = TransformerFactory.newInstance();
Transformer idTransform = xformFactory.newTransformer();
idTransform.transform(input, output);
Node document = idTransform.getNode();

The transform will walk the DOM tree firing off events to the SAX ContentHandler.

Although TrAX is the most standard, parser-independent means of passing documents back and forth between SAX and DOM, many implementations of these APIs also provide their own utility classes for crossing the border between the APIs, For example, GNU JAXP has the gnu.xml.pipeline.DomConsumer class for building DOM Document objects from SAX event streams and the gnu.xml.util.DomParser class for feeding a DOM Document into a SAX program. The Oracle XML Parser for Java provides the oracle.xml.parser.v2.DocumentBuilder is a SAX ContentHandler/LexicalHandler/DeclHandler that builds a DOM Document from a SAX XMLReader.

Prev | Up | Next
DOMException | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 29, 2002


Summary
Prev Chapter 9. The Document Object Model Next

Summary

The Document Object Model, DOM, is a W3C standard API for reading and writing XML and HTML documents represented as trees. DOM is defined in IDL, but there are standard bindings for Java. In DOM, an XML document is represented as a connected tree of Node objects. The root of the tree is a Document object. Other kinds of nodes found in the tree include Element, Text, Comment, ProcessingInstruction, and several more. The basic Node interface provides generic methods to navigate the tree, as well as get the names, values, local names, prefixes, types, and namespace URIs of each node. Not all these properties really make sense for all kinds nodes so many of these methods can return null.

DOM Level 2 does not provide any standard way to parse an existing document, serialize a document onto a stream, or load the parser’s DOMImplementation. Sun’s Java API for XML parsing (JAXP) fills these holes. DOM Level 3 will also add this functionality as a standard part of DOM.

Prev | Up | Next
Choosing between SAX and DOM | Home | Chapter 10. Creating XML Documents with DOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified December 17, 2001


Chapter 10. Creating XML Documents with DOM
Prev Next

Chapter 10. Creating XML Documents with DOM

Table of Contents

DOMImplementation
Locating a DOMImplementation
Implementation Specific Class
JAXP DocumentBuilder
DOM3 DOMImplementationRegistry
The Document Interface as an Abstract Factory
The Document Interface as a Node Type
Getter methods
Finding elements
Transferring nodes between documents
Normalization
Summary

DOM is a read-write API. DOM documents are created not only by parsing text files, but also by creating new documents in memory out of nothing at all. These documents can then be serialized onto a stream or into a file. The abstract factory interface that creates new Document objects is called DOMImplementation. The Document interface has a dual purpose. First it represents XML documents themselves and provides access to their contents, document type declaration, and other properties. Secondly it too is an abstract factory responsible for creating the nodes that go in the document: elements, text, comments, processing instructions, etc. Each such node belongs exclusively to the document that created it and cannot be moved to a different document.

DOMImplementation

The DOMImplementation interface, shown in Example 10.1, is an abstract factory that is responsible for creating two things, new Document and DocumentType objects. It also provides the hasFeature() method discussed in the last chapter that tells you what features this implementation supports.

Example 10.1. The DOMImplementation interface

package org.w3c.dom;

public interface DOMImplementation {

  public DocumentType createDocumentType(
   String rootElementQualifiedName,
   String publicID, String systemID) throws DOMException;
  public Document createDocument(String rootElementNamespaceURI,
   String rootElementQualifiedName, DocumentType doctype)
   throws DOMException;
  public boolean hasFeature(String feature, String version);

}

For example, given a DOMImplementation object named impl, this chunk of code creates a new DocumentType object named svgDOCTYPE pointing to the Scalable Vector Graphics DTD:

DocumentType svgDOCTYPE = impl.createDocumentType("svg",
 "-//W3C//DTD SVG 1.0//EN",
 "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd");

If the DTD does not have a public ID, you can just pass null for the second argument.

You can use this DocumentType object when constructing a new SVG Document object:

Document svgDoc = impl.createDocument(
 "http://www.w3.org/2000/svg", "svg", svgDOCTYPE
);

If svgDoc were serialized into a text file, it would look something like this (modulo insignificant white space):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN"
 "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
<svg xmlns="http://www.w3.org/2000/svg"/>

Of course not all XML documents have DOCTYPE declarations or namespace URIs. If the document is merely well-formed, then you can just pass null for the doctype argument. If the document root element is not in a namespace, you can also pass null for the namespace URI. This code fragment creates an XML-RPC document with neither a document type declaration nor a namespace URI:

Document xmlrpc = impl.createDocument(null, "methodCall", null);

These Document objects, with or without DOCTYPE declarations, are not yet complete. In particular, they do not yet have any content beyond an empty root element. For that, you’ll have to use the methods of the Document interface to create nodes, and the methods of the Node interface to add these newly created nodes to the tree.

Prev | Up | Next
Summary | Home | Locating a DOMImplementation
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 27, 2002


Locating a DOMImplementation
Prev Chapter 10. Creating XML Documents with DOM Next

Locating a DOMImplementation

So far, I’ve deliberately avoided the crucial question of how one creates a DOMImplementation object in the first place. Since DOMImplementation is an interface, not a class, it cannot be instantiated directly through its own constructor. Instead you have to build it in one of three ways:

  • Construct the implementation specific class

  • Use the JAXP DocumentBuilder factory class.

  • Use the DOM Level 3 DOMImplementationRegistry factory class.

Implementation Specific Class

Directly constructing an instance of the vendor class that implements DOMImplementation is the simplest of the three alternatives. However, the name of this class and how it’s created vary from one implementation to the next. For example, in Xerces the org.apache.xerces.dom.DOMImplementationImpl singleton class implements the DOMImplementation interface. The singleton object is retrieved via the getDOMImplementation() factory method like this:

DOMImplementation impl
 = DOMImplementationImpl.getDOMImplementation();

However, if you were to switch to a different implementation, you’d need to change your source code and recompile. For example, in the Oracle XML parser for Java, oracle.xml.parser.v2.XMLDOMImplementation class implements the DOMImplementation interface and instances of this class are created with a no-args constructor, like this:

DOMImplementation impl = new XMLDOMImplementation();

In both cases, the implementation specific object is assigned to a variable of type DOMImplementation. This allows the compiler to make sure you don’t accidentally use any implementation-specific methods in the object, or tie the code too tightly to one vendor. The implementation-dependent code should be limited to this one line.

JAXP DocumentBuilder

The JAXP DocumentBuilder class introduced in the last chapter has a getDOMImplementation() method that can locate a local DOMImplementation class.

public abstract DOMImplementation getDOMImplementation();

For example, this code fragment uses JAXP to create a new SVG Document object in memory:

try {
  DocumentBuilderFactory factory
   = DocumentBuilderFactory.newInstance();
  DocumentBuilder builder = factory.newDocumentBuilder();
  DOMImplementation impl = builder.getDOMImplementation();

  DocumentType svgDOCTYPE = impl.createDocumentType("svg",
   "-//W3C//DTD SVG 1.0//EN",
   "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd");
  Document svgDoc = impl.createDocument(
   "http://www.w3.org/2000/svg", "svg", svgDOCTYPE
  );
  // work with the document...
}
catch (FactoryConfigurationError e) {
  System.out.println(
   "Could not locate a JAXP DocumentBuilderFactory class");
}
catch (ParserConfigurationException e) {
  System.out.println(
   "Could not locate a JAXP DocumentBuilder class");
}

If you only want to create a new Document, then DocumentBuilder also has a newDocument() method that shortcuts the DOMImplementation class:

public abstract Document newDocument();

However, this method does not properly set a root element. Thus the documents created by newDocument() are at least initially malformed; and therefore I recommend that you don’t use this method. Just use JAXP to retrieve a DOMImplementation object and use its createDocument() method instead.

The specific implementation JAXP chooses is determined in the same way as described for locating a parser in the last chapter. That is, first JAXP reads the javax.xml.parsers.DocumentBuilderFactory system property, then the lib/jaxp.properties file, then the META-INF/services/javax.xml.parsers.DocumentBuilderFactory file in all JAR files available to the runtime, and then finally a fallback class hardcoded into the JAXP implementation.

DOM3 DOMImplementationRegistry

The final option for locating the DOMImplementation is new in DOM Level 3 and only supported by Xerces 2 so far. This is the DOMImplementationRegistry class shown in Example 10.2.

Example 10.2. The DOMImplementationRegistry class

package org.w3c.dom;

public class DOMImplementationRegistry  {

  // The system property that specifies DOMImplementationSource
  // class names.
  public static String PROPERTY
   = "org.w3c.dom.DOMImplementationSourceList";

  public static DOMImplementation getDOMImplementation(
   String features) throws ClassNotFoundException,
   InstantiationException, IllegalAccessException;

  public static void addSource(DOMImplementationSource s)
   throws ClassNotFoundException, InstantiationException,
   IllegalAccessException;

}

The getDOMImplementation() method returns a DOMImplementation object that supports the features given in the argument, or null if no such implementation can be found. For example, this code fragment requests a DOMImplementation that supports XML DOM Level 1, any version of the traversal module, and DOM Level 2 events:

try {
  DOMImplementation impl = DOMImplementationRegistry
   .getDOMImplementation("XML 1.0 Traversal Events 2.0");
  if (impl != null) {
    DocumentType svgDOCTYPE = impl.createDocumentType("svg",
     "-//W3C//DTD SVG 1.0//EN",
     "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd");
    Document svgDoc = impl.createDocument(
     "http://www.w3.org/2000/svg", "svg", svgDOCTYPE
    );
    // work with the document...
  }
}
catch (Exception e) {
  System.out.println(e);
}

Be sure to check whether the implementation returned is null before using it. Many installations may not be able to support all the features you ask for.

DOMImplementationRegistry searches for DOMImplementation classes by looking at the value of the org.w3c.dom.DOMImplementationSourceList Java system property. This property should contain a white space separated list of DOMImplementationSource classes on the local system. This interface is summarized in Example 10.3.

Example 10.3. The DOMImplementationSource interface

package org.w3c.dom;

public interface DOMImplementationSource {

  public DOMImplementation getDOMImplementation(String features);

}

DOMImplementationRegistry.getDOMImplementation() queries each source for its DOMImplementation. The double indirection (listing DOMImplementationSource classes rather than DOMImplementation classes) is necessary to allow DOMImplementationRegistry.getDOMImplementation() to return different classes of objects depending on which combination of features are requested.

The three exceptions that getDOMImplementation() throws— ClassNotFoundException, InstantiationException, and IllegalAccessException— shouldn’t be very common. The only way any of these can be thrown is if the org.w3c.dom.DOMImplementationSourceList system property includes the name of a class that can’t be found or one that is not a conforming instance of DOMImplementationSource.

However, like the other DOM Level 3 material discussed in this book, all of this is on the wrong side of the bleeding edge and cannot be expected to work in most existing implementations.

Prev | Up | Next
Chapter 10. Creating XML Documents with DOM | Home | The Document Interface as an Abstract Factory
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 13, 2002


The Document Interface as an Abstract Factory
Prev Chapter 10. Creating XML Documents with DOM Next

The Document Interface as an Abstract Factory

The Document interface, summarized in Example 10.4, serves two purposes in DOM:

  • An abstract factory that creates instances of other nodes for that document

  • The representation of the document node

Example 10.4. The Document interface

package org.w3c.dom;

public interface Document extends Node {

  public Element createElement(String tagName)
   throws DOMException;
  public Element createElementNS(String namespaceURI,
   String qualifiedName) throws DOMException;
  public Text createTextNode(String data);
  public Comment createComment(String data);
  public CDATASection createCDATASection(String data)
   throws DOMException;
  public ProcessingInstruction createProcessingInstruction(
   String target, String data) throws DOMException;
  public Attr createAttribute(String name) throws DOMException;
  public Attr createAttributeNS(String namespaceURI,
   String qualifiedName) throws DOMException;
  public DocumentFragment createDocumentFragment();
  public EntityReference createEntityReference(String name)
   throws DOMException;

  public DocumentType      getDoctype();
  public DOMImplementation getImplementation();
  public Element           getDocumentElement();
  public Node              importNode(Node importedNode,
                               boolean deep) throws DOMException;
  public NodeList          getElementsByTagName(String tagname);
  public NodeList          getElementsByTagNameNS(
                          String namespaceURI, String localName);
  public Element           getElementById(String elementId);

}

Don’t forget that besides the methods listed here, each Document object also has all the methods of the Node interface discussed in the last chapter. These are key parts of the functionality of the class.

I’ll begin with its use as an abstract factory. You’ll notice that the Document interface has nine separate createXXX() methods for creating seven different kinds of node objects. (There are two methods each for creating element and attribute nodes because you can create these with or without namespaces.) For example, given a Document object doc, this code fragment creates a new processing instruction and a comment:

ProcessingInstruction xmlstylesheet
 = doc.createProcessingInstruction("xml-stylesheet",
 "type=\"text/css\" href=\"standard.css\"");
Comment comment = doc.createComment(
 "An example from Chapter 10 of Processing XML with Java");

Although these two nodes are associated with the document, they are not yet parts of its tree. To add them, it’s necessary to use the insertBefore() method of the Node interface which Document extends. Specifically, I’ll insert each of these nodes before the root element of the document, which can be retrieved via getDocumentElement():

Node rootElement = doc.getDocumentElement();
doc.insertBefore(comment, rootElement);
doc.insertBefore(xmlstylesheet, rootElement);

To add content inside the root element, it’s necessary to use the Node methods on the root element. For example, this code fragment, adds a desc child element to the root element:

Element desc
 = doc.createElementNS("http://www.w3.org/2000/svg", "desc");
rootElement.appendChild(desc);

Each node is created by the owner document. However, it is inserted using the parent node. For example, this code fragment adds a text node child containing the words “An example from Processing XML with Java” to the previous desc element node:

Text descText
 = doc.createTextNode("An example from Processing XML with Java");
desc.appendChild(descText);

Example 10.5 puts this all together to create a program that builds a complete, albeit very simple, SVG document in memory using DOM. JAXP loads the DOMImplementation so that the program is reasonably parser-independent. The JAXP ID-transform hack introduced in the last chapter dumps the document on System.out.

Example 10.5. Building an SVG document in memory using DOM

import javax.xml.parsers.*;
import javax.xml.transform.*;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.dom.DOMSource;
import org.w3c.dom.*;
import java.io.IOException;

public class SimpleSVG {

  public static void main(String[] args) {

    try {
      // Find the implementation
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setNamespaceAware(true);
      DocumentBuilder builder = factory.newDocumentBuilder();
      DOMImplementation impl = builder.getDOMImplementation();

      // Create the document
      DocumentType svgDOCTYPE = impl.createDocumentType(
       "svg", "-//W3C//DTD SVG 1.0//EN",
       "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"
      );
      Document doc = impl.createDocument(
       "http://www.w3.org/2000/svg", "svg", svgDOCTYPE);

      // Fill the document
      Node rootElement = doc.getDocumentElement();
      ProcessingInstruction xmlstylesheet
       = doc.createProcessingInstruction("xml-stylesheet",
       "type=\"text/css\" href=\"standard.css\"");
      Comment comment = doc.createComment(
       "An example from Chapter 10 of Processing XML with Java");
      doc.insertBefore(comment, rootElement);
      doc.insertBefore(xmlstylesheet, rootElement);
      Node desc = doc.createElementNS(
       "http://www.w3.org/2000/svg", "desc");
      rootElement.appendChild(desc);
      Text descText = doc.createTextNode(
       "An example from Processing XML with Java");
      desc.appendChild(descText);

      // Serialize the document onto System.out
      TransformerFactory xformFactory
       = TransformerFactory.newInstance();
      Transformer idTransform = xformFactory.newTransformer();
      Source input = new DOMSource(doc);
      Result output = new StreamResult(System.out);
      idTransform.transform(input, output);

    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a JAXP factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println(
        "Could not locate a JAXP DocumentBuilder class"
      );
    }
    catch (DOMException e) {
      System.err.println(e);
    }
    catch (TransformerConfigurationException e) {
      System.err.println(e);
    }
    catch (TransformerException e) {
      System.err.println(e);
    }

  }

}

When this program is run, it produces the following output:

C:\XMLJAVA>java SimpleSVG
<?xml version="1.0" encoding="utf-8"?><!--An example from Chapter
10 of Processing XML with Java--><?xml-stylesheet type="text/css"
href="standard.css"?><svg><desc>An example from Processing XML
with Java</desc></svg>

I’ve inserted line breaks to make the output fit on this page. However, the actual output doesn’t have any. In the prolog, that’s because the JAXP ID transform doesn’t include any. In the document, that’s because the program did not add any text nodes containing only white space. Many parser vendors include custom serialization packages that allow you to more closely manage the placement of white space and other syntax sugar in the output. In addition, this will be a standard part of DOM3. We’ll explore these options for prettifying the output in Chapter 13.

Note

The lack of namespace declarations and possibly the lack of a DOCTYPE declaration is a result of bugs in JAXP implementations. I’ve reported the problem to several XSLT processor/XML parser vendors and am hopeful that at least some of them will fix this bug before the final draft of this book. As of May, 2002 GNU JAXP and Oracle include the namespace declaration while Xerces 2.0.1 leaves it out. So far no implementation I've seen includes the DOCTYPE declaration.

The same techniques can be used for all the nodes in the tree: text, comments, elements, processing instructions, and entity references. Attributes are not children though. Attribute nodes can only be set on element nodes and only by using the methods of the Element interface. I’ll take that up in the next chapter. However, Attr objects are created by Document objects, just like all the other DOM node objects.

DOM is not picky about whether you work from the top down or the bottom up. You can start at the root, and add its children; then add the child nodes to these nodes, and continue on down the tree. Alternately, you can start by creating the deepest nodes in the tree, and then create their parents, and then the parents of the parents, and so on back up to the root. Or you can mix and match as seems appropriate in your program. DOM really doesn’t care as long as there’s always a root element.

Each node that’s created is firmly associated with the document that created it. If document A creates node X, then node X cannot be inserted into document B. A copy of node X can be imported into document B, but node X itself is always attached only to document A.

We’re now in a position to repeat some examples from Chapter 3 but this time using DOM to create the document rather than just writing strings onto a stream. Among other advantages this means that many well-formedness constraints are automatically satisfied. Furthermore, the programs will have a much more object oriented feel to them.

I’ll begin with the simple Fibonacci problem of Example 3.3. That program produced documents that look like this:

<?xml version="1.0"?>
<Fibonacci_Numbers>
  <fibonacci>1</fibonacci>
  <fibonacci>1</fibonacci>
  <fibonacci>2</fibonacci>
  <fibonacci>3</fibonacci>
  <fibonacci>5</fibonacci>
  <fibonacci>8</fibonacci>
  <fibonacci>13</fibonacci>
  <fibonacci>21</fibonacci>
  <fibonacci>34</fibonacci>
  <fibonacci>55</fibonacci>
</Fibonacci_Numbers>

This is a straightforward element based hierarchy that does not use namespaces or document type declarations. Although simple, these sorts of documents are important. XML-RPC is just one of many real-world applications that does not use anything more than element, text, and document nodes.

Example 10.6 is a DOM-based program that generates documents of this form. It is at least superficially more complex than the equivalent program from Chapter 3. However, it has some advantages that program does not. In particular, well-formedness of the output is almost guaranteed. It’s a lot harder to produce incorrect XML with DOM than simply by writing strings on a stream. Secondly, the data structure is a lot more flexible. Here, the document is written more or less from beginning to end. However, if this were part of a larger program that ran for a longer period of time, nodes could be added and deleted in almost random order anywhere in the tree at any time. It’s not necessary to know all the information that will ever go into the document before you begin writing it. The downside to this is that DOM programs tend to eat substantially more RAM than the streaming equivalents because they have to keep the entire document in memory at all times. This can be a significant problem for large documents.

Example 10.6. A DOM program that outputs the Fibonacci numbers as an XML document

import org.w3c.dom.*;
import javax.xml.parsers.*;
import javax.xml.transform.*;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import java.math.BigInteger;

public class FibonacciDOM {

  public static void main(String[] args) {

    try {

      // Find the implementation
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setNamespaceAware(true);
      DocumentBuilder builder = factory.newDocumentBuilder();
      DOMImplementation impl = builder.getDOMImplementation();

      // Create the document
      Document doc = impl.createDocument(null,
       "Fibonacci_Numbers", null);

      // Fill the document
      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      Element root = doc.getDocumentElement();

      for (int i = 0; i < 10; i++) {
        Element number = doc.createElement("fibonacci");
        Text text = doc.createTextNode(low.toString());
        number.appendChild(text);
        root.appendChild(number);

        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }

      // Serialize the document onto System.out
      TransformerFactory xformFactory
       = TransformerFactory.newInstance();
      Transformer idTransform = xformFactory.newTransformer();
      Source input = new DOMSource(doc);
      Result output = new StreamResult(System.out);
      idTransform.transform(input, output);

    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a JAXP factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println(
        "Could not locate a JAXP DocumentBuilder class"
      );
    }
    catch (DOMException e) {
      System.err.println(e);
    }
    catch (TransformerConfigurationException e) {
      System.err.println(e);
    }
    catch (TransformerException e) {
      System.err.println(e);
    }

  }

}

As usual, this code is broken up into the four main parts of creating a new XML document with DOM:

  1. Locate a DOMImplementation

  2. Create a new Document object.

  3. Fill the Document with various kinds of nodes.

  4. Serialize the Document onto a stream.

Most DOM programs that create new documents follow this structure. They may hide the different parts in different methods, or use DOM3 to serialize instead of JAXP; but they all have to locate a DOMImplementation, use that to create a Document object, fill the document with other nodes created by the Document object, and then finally serialize the result. (A few programs may occasionally skip the serialization step.)

The only part that really changes from one program to the next is how the document is filled with content. This naturally depends on the structure of the document. A program that reads tables from a database to get the data will naturally look very different from a program like this one that algorithmically generates numbers. And both of these will look very different from a program that asks the user to type in information. However, all three and many more besides will use the same methods of the Document and Node interfaces to build the structures they need.

Here’s the output when this program is run:

C:\XMLJAVA>java FibonacciDOM
<?xml version="1.0" encoding="utf-8"?><Fibonacci_Numbers>
<fibonacci>1</fibonacci><fibonacci>1</fibonacci><fibonacci>2
</fibonacci><fibonacci>3</fibonacci><fibonacci>5</fibonacci>
<fibonacci>8</fibonacci><fibonacci>13</fibonacci><fibonacci>21
</fibonacci><fibonacci>34</fibonacci><fibonacci>55</fibonacci>
</Fibonacci_Numbers>

You see once again that the white space is not quite what was expected. One way to fix this is to add the extra text nodes that represent the white space. For example,

      for (int i = 0; i < 10; i++) {
        Text space = doc.createTextNode("\n  ");
        root.appendChild(space);
        Element number = doc.createElement("fibonacci");
        Text text = doc.createTextNode(low.toString());
        number.appendChild(text);
        root.appendChild(number);

        BigInteger temp = high;
        high = high.add(low);
        low  = temp;
      }
      Text lineBreak = doc.createTextNode("\n");
      root.appendChild(lineBreak);

Alternately, you can use a more sophisticated serializer and tell it to add the extra white space. I prefer this approach because it’s much simpler and does not clutter up the code with basically insignificant white space. I’ll demonstrate this in Chapter 13. Of course, if you really do care about white space, then you need to manage the white-space only text nodes explicitly and tell whichever serializer you use to leave the white space alone.

Adding namespaces or a DOCTYPE declaration pointing to an external DTD subset, is not significantly harder. For example, suppose you want to generate valid MathML like Example 10.7:

Example 10.7. A valid MathML document containing Fibonacci numbers

<?xml version="1.0"?>
<!DOCTYPE math PUBLIC "-//W3C//DTD MathML 2.0//EN"
 "http://www.w3.org/TR/MathML2/dtd/mathml2.dtd">
<math xmlns:mathml="http://www.w3.org/1998/Math/MathML">
  <mrow><mi>f(1)</mi><mo>=</mo><mn>1</mn></mrow>
  <mrow><mi>f(2)</mi><mo>=</mo><mn>1</mn></mrow>
  <mrow><mi>f(3)</mi><mo>=</mo><mn>2</mn></mrow>
  <mrow><mi>f(4)</mi><mo>=</mo><mn>3</mn></mrow>
  <mrow><mi>f(5)</mi><mo>=</mo><mn>5</mn></mrow>
  <mrow><mi>f(6)</mi><mo>=</mo><mn>8</mn></mrow>
  <mrow><mi>f(7)</mi><mo>=</mo><mn>13</mn></mrow>
  <mrow><mi>f(8)</mi><mo>=</mo><mn>21</mn></mrow>
  <mrow><mi>f(9)</mi><mo>=</mo><mn>34</mn></mrow>
  <mrow><mi>f(10)</mi><mo>=</mo><mn>55</mn></mrow>
</math>

The markup is somewhat more complex, but the Java code is not significantly more so. You simply need to use the implementation to create a new DocumentType object, and include both that and the namespace URL in the call to createDocument(). Example 10.8 demonstrates.

Example 10.8. A DOM program that outputs the Fibonacci numbers as a MathML document

import org.w3c.dom.*;
import javax.xml.parsers.*;
import javax.xml.transform.*;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import java.math.BigInteger;

public class FibonacciMathMLDOM {

  public static void main(String[] args) {

    try {

      // Find the implementation
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setNamespaceAware(true);
      DocumentBuilder builder = factory.newDocumentBuilder();
      DOMImplementation impl = builder.getDOMImplementation();

      // Create the document
      DocumentType mathml = impl.createDocumentType("math",
       "-//W3C//DTD MathML 2.0//EN",
       "http://www.w3.org/TR/MathML2/dtd/mathml2.dtd");
      Document doc = impl.createDocument(
       "http://www.w3.org/1998/Math/MathML", "math", mathml);

      // Fill the document
      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      Element root = doc.getDocumentElement();

      for (int i = 1; i <= 10; i++) {
        Element mrow = doc.createElement("mrow");

        Element mi = doc.createElement("mi");
        Text function = doc.createTextNode("f(" + i + ")");
        mi.appendChild(function);

        Element mo = doc.createElement("mo");
        Text equals = doc.createTextNode("=");
        mo.appendChild(equals);

        Element mn = doc.createElement("mn");
        Text value = doc.createTextNode(low.toString());
        mn.appendChild(value);

        mrow.appendChild(mi);
        mrow.appendChild(mo);
        mrow.appendChild(mn);

        root.appendChild(mrow);

        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }

      // Serialize the document onto System.out
      TransformerFactory xformFactory
       = TransformerFactory.newInstance();
      Transformer idTransform = xformFactory.newTransformer();
      Source input = new DOMSource(doc);
      Result output = new StreamResult(System.out);
      idTransform.transform(input, output);

    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a JAXP factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println(
        "Could not locate a JAXP DocumentBuilder class"
      );
    }
    catch (DOMException e) {
      System.err.println(e);
    }
    catch (TransformerConfigurationException e) {
      System.err.println(e);
    }
    catch (TransformerException e) {
      System.err.println(e);
    }

  }

}

Internal DTD subsets are a little harder though, and not really supported at all in DOM2. For example, let’s suppose you want to use a namespace prefix on your MathML elements, but still have the document be valid MathML. The MathML DTD is designed in such a way that you can change the prefix and whether or not prefixes are used by redefining the MATHML.prefixed and MATHML.prefix parameter entities. For instance, Example 10.9 uses the prefix math:

Example 10.9. A valid MathML document using prefixed names

<?xml version="1.0"?>
<!DOCTYPE math:math PUBLIC "-//W3C//DTD MathML 2.0//EN"
 "http://www.w3.org/TR/MathML2/dtd/mathml2.dtd" [
  <!ENTITY % MATHML.prefixed "INCLUDE">
  <!ENTITY % MATHML.prefix "math">
]>
<math:math xmlns:mathml="http://www.w3.org/1998/Math/MathML">
  <math:mrow>
    <math:mi>f(1)</math:mi>
    <math:mo>=</math:mo>
    <math:mn>1</math:mn>
  </math:mrow>
  <math:mrow>
    <math:mi>f(2)</math:mi>
    <math:mo>=</math:mo>
    <math:mn>1</math:mn>
  </math:mrow>
  <math:mrow>
    <math:mi>f(3)</math:mi>
    <math:mo>=</math:mo>
    <math:mn>2</math:mn>
  </math:mrow>
  <math:mrow>
    <math:mi>f(4)</math:mi>
    <math:mo>=</math:mo>
    <math:mn>3</math:mn>
  </math:mrow>
</math:math>

Using prefixed names in DOM code is straightforward enough. However, there’s no way to override the entity definitions in the DTD to tell it to validate against the prefixed names. DOM does not provide any means to create a new internal DTD subset or change an existing one. This means that in order for the document you generate to be valid, it must use the same prefix the DTD does.

There are some hacks that can work around this. Some of the concrete classes that implement the DocumentType interface such as Xerces’s org.apache.xerces.dom.DocumentTypeImpl include a non-standard setInternalSubset() method. Or instead of pointing to the normal DTD, you can point to an external DTD that overrides the namespace parameter entity references and then imports the usual DTD. You could even generate this DTD on the fly using a separate output stream that writes strings containing entity declarations into a file. However, the bottom line is that the internal DTD subset just isn’t well supported by DOM, and any program that needs access to it should use a different API.

Prev | Up | Next
Locating a DOMImplementation | Home | The Document Interface as a Node Type
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 12, 2002


The Document Interface as a Node Type
Prev Chapter 10. Creating XML Documents with DOM Next

The Document Interface as a Node Type

Besides the factory methods and the methods common to all nodes, the Document interface has several unique methods that perform operations relevant only to document nodes. These include:

  • Getter methods

  • Methods to find elements

  • A method to copy nodes from other documents

Getter methods

The Document interface has three methods that simply return particular parts of the document:

public Element getDocumentElement();
public DocumentType getDoctype();
public DOMImplementation getImplementation();

These are fairly self-explanatory. You’ve already seen the getDocumentElement() method used several times. It just returns the Element object representing the root element of the document. Similarly, the getDoctype() method returns the document’s DocumentType object or null if the document does not have a document type declaration. The getImplementation() method returns the DOMImplementation object that created this document.

Several pieces are missing. In particular, no part of the XML declaration is available: not version, not encoding, not standalone status. Other useful information that’s missing from DOM2 includes the actual encoding of the document (which is usually but not always the same as the encoding declared in the XML declaration) and the base URI of the document against which relative URIs in the document should be resolved. DOM Level 3 will add several more getter and setter methods to the Document interface to make these available.

public String getActualEncoding();
public void setActualEncoding(String actualEncoding);
public String getEncoding();
public void setEncoding(String encoding);
public boolean getStandalone();
public void setStandalone(boolean standalone);
public String getVersion();
public void setVersion(String version);
public void setBaseURI(String baseURI)
    throws DOMException;

The obvious getBaseURI() is not really missing. It’s just included in the Node super-interface rather than directly in the Document interface. Thus you can find out the base URI for any kind of node. This is important because XML documents can be built from multiple entities and different nodes may come from different files.

public String getBaseURI();

Finally DOM Level 3 adds one more setter/getter pair that, strictly speaking, doesn’t describe the document so much as the implementation. These two methods determine how draconian DOM is about checking for errors as the document is built in memory:

public boolean getStrictErrorChecking();
public void setStrictErrorChecking(boolean strictErrorChecking);

If the strict error checking property is false, then the implementation may not make every test it could possibly make. For instance, it might allow namespace prefixes that are not mapped to namespace URIs or make a text node a child of the document element. This can be faster, but it is also dangerous since other code may fail when presented with a Document object that does not satisfy all the usual constraints. Strict error checking is enabled by default. Even if strict error checking is false, however, some error checking may still be done. The purpose of these methods is to allow implementations to skip some of the tedious checking they normally do and thus improve performance. The purpose is not to allow malformed documents, though that may be the effect in some cases.

These properties are experimentally supported in Xerces 2.0.2. No other parsers support them at the time of this writing. The detailed signatures are still subject to change though, and you should not rely on them.

Example 10.10 is a simple program that parses a document from a URL passed through the command line and prints the values of these various properties. Since Xerces is currently the only parser to support the DOM3 properties, I used its implementation classes explicitly rather than the more generic JAXP.

Example 10.10. The properties of a Document object

import org.apache.xerces.parsers.DOMParser;
import org.apache.xerces.dom.DocumentImpl;
import org.w3c.dom.*;
import org.xml.sax.SAXException;
import java.io.IOException;

public class DocumentProperties {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java DocumentProperties URL");
      return;
    }
    String url = args[0];

    // Since this only works in Xerces 2.0.2, we might as well use the
    // Xerces-specific implementation classes instead of JAXP.
    DOMParser parser = new DOMParser();
    try {
      parser.parse(url);
      DocumentImpl document = (DocumentImpl) parser.getDocument();

      // DOM2 properties
      System.out.println("Implementation: " + document.getImplementation());
      System.out.println("Root element: " + document.getDocumentElement());
      System.out.println("DOCTYPE: " + document.getDoctype());

      // DOM3 Properties
      System.out.println("Version: " + document.getVersion());
      System.out.println("Standalone: " + document.getStandalone());
      System.out.println("Declared encoding: " + document.getEncoding());
      System.out.println("Strict error checking: " + document.getStrictErrorChecking());
      System.out.println("Actual encoding: " + document.getActualEncoding());
      System.out.println("Base URI: " + document.getBaseURI());

    }
    catch (SAXException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not read " + url
      );
    }

  }

}

Here’s the output from when I ran this program against the DocBook source code for this chapter:

$ java DocumentProperties ch10.xml
Implementation: org.apache.xerces.dom.DOMImplementationImpl@ef9f1d
Root element: [chapter: null]
DOCTYPE: [chapter: null]
Version: 1.0
Standalone: false
Declared encoding: UTF-8
Strict error checking: true
Actual encoding: UTF-8
Base URI: file:///home/elharo/books/xmljava/ch10.xml

In this case, the detailed output depends on what the toString() method for each of the implementation classes does. Even once other parsers support DOM3, this will be something different for different parsers. A more serious application would use the methods of each interface (Document, Doctype, and Element) to provide more complete output.

Tip

If a non-Xerces DOM implementation precedes Xerces in your class path, then this program won’t compile. You need to make sure Xerces is the first DOM the compiler and runtime find. This is especially problematic in Java 1.4, which includes a DOM implementation that does not support the DOM3 properties used here. However, in Java 1.4 you can use the java interpreter’s -Xbootclasspath/p: option to prepend JAR archives to the boot classpath so that they will be preferred to the ones bundled with Java.

Finding elements

Some of the most useful methods in the Document interface are those that retrieve all the elements with certain names or IDs in the document, irrespective of where in the document they may actually be. When you’re only really interested in certain elements, this can avoid a lot of tedious and complex tree-walking. These three methods are:

public NodeList getElementsByTagName(String tagName);
public NodeList getElementsByTagNameNS(String namespaceURI, String localName);
public Element getElementByID();

The first two methods return a NodeList of the elements with the specified name or local name/namespace URI pair. This list is in document order. You can use the asterisk (*) to match all names or all namespace URIs.

The third method returns the single element with the specified ID value, or null if no such element is present in the document. The ID is given by an ID-type attribute on that element.

Caution

It is possible though invalid for multiple elements in one document to share the same ID. In this case, this method’s behavior is undefined. For maximum safety, you may want to limit this method to provably valid documents.

As a demonstration, let’s develop a an XML-RPC servlet that generates Fibonacci numbers. (This is actually what was on the other side of the clients you saw in Chapter 5 and Chapter 3). Recall that the request document looks like Example 10.11. The server needs to find the integer value of the single param. Since we know there’s exactly one int element in the request, its easy to use getElementsByTagName() to find it.

Example 10.11. An XML-RPC request document

<?xml version="1.0"?>
<methodCall>
  <methodName>calculateFibonacci</methodName>
  <params>
    <param>
      <value><int>23</int></value>
    </param>
  </params>
</methodCall>

The server needs to calculate the result based on the input transmitted by the client, wrap that up in a response document like the one shown in Example 10.12, and transmit that document back to the client.

Example 10.12. An XML-RPC response document

<?xml version="1.0"?>
<methodResponse>
  <params>
    <param>
      <value><double>28657</double></value>
    </param>
  </params>
</methodResponse>

Example 10.13 shows the complete servlet. It extends HttpServlet and implements SingleThreadModel. This interface lets the servlet container know that this servlet is not thread safe, and it should use a different instance of this class for each concurrent thread. However, one instance may be used for successive threads. This was necessary here because the JAXP DocumentBuilder and Transformer classes and possibly the class that implements DOMImplementation are not thread-safe. You could make the servlet thread safe by loading new instances of these interfaces inside doPost() rather than sharing instances created in init(). However, in a potentially high-volume server environment the resource cost for that feels disturbingly large. A better alternative would be to manually synchronize access to these objects inside doPost(). However, since proper synchronization is notoriously difficult, I prefer to leave the work to the server.

Example 10.13. A DOM based XML-RPC servlet

import javax.servlet.*;
import javax.servlet.http.*;
import java.io.*;
import java.math.BigInteger;
import org.w3c.dom.*;
import org.xml.sax.SAXException;
import javax.xml.parsers.*;
import javax.xml.transform.*;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;

public class FibonacciXMLRPCDOMServlet extends HttpServlet
 implements SingleThreadModel {

  // Fault codes
  public final static int MALFORMED_REQUEST_DOCUMENT = 1;
  public final static int INVALID_REQUEST_DOCUMENT   = 2;
  public final static int INDEX_MISSING              = 3;
  public final static int NON_POSITIVE_INDEX         = 4;
  public final static int BAD_INTEGER_FORMAT         = 5;
  public final static int UNEXPECTED_PROBLEM         = 255;

  private DocumentBuilder   parser;
  private DOMImplementation impl;
  private Transformer       idTransform;

  // Load a parser, transformer, and implementation
  public void init() throws ServletException {

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      this.parser = factory.newDocumentBuilder();
      this.impl   = parser.getDOMImplementation();
    }
    catch (Throwable t) {
      // It's unusual to catch a generic Throwable instead of an
      // exception. Here I'm specifically worried about
      // FactoryConfigurationErrors and
      // ParserConfigurationExceptions, both of which are real
      // possibilities in a servlet environment because of the
      // weird ways servlet containers arrange classpaths.
      throw new ServletException(
       "Could not locate a JAXP parser", t);
    }

    try {
      TransformerFactory xformFactory
       = TransformerFactory.newInstance();
      this.idTransform = xformFactory.newTransformer();
    }
    catch (Throwable t) {
      throw new ServletException(
       "Could not locate a JAXP transformer", t);
    }

  }

  // Respond to an XML-RPC request
  public void doPost(HttpServletRequest servletRequest,
   HttpServletResponse servletResponse)
   throws ServletException, IOException {

    servletResponse.setContentType("text/xml; charset=UTF-8");
    PrintWriter out = servletResponse.getWriter();
    InputStream in  = servletRequest.getInputStream();

    Document request;
    Document response;
    try {
      request = parser.parse(in);

      NodeList ints = request.getElementsByTagName("int");
      if (ints.getLength() == 0) {
        // XML-RPC allows i4 as an alias for int.
        ints = request.getElementsByTagName("i4");
      }
      Node input = ints.item(0); // throws NullPointerException
      String generations = getFullText(input);
      int numberOfGenerations = Integer.parseInt(generations);
      BigInteger result = calculateFibonacci(numberOfGenerations);
      response = makeResponseDocument(result);
    }
    catch (SAXException e) {
      response = makeFaultDocument(MALFORMED_REQUEST_DOCUMENT, e.getMessage());
    }
    catch (NullPointerException e) {
      response = makeFaultDocument(INDEX_MISSING, e.getMessage());
    }
    catch (NumberFormatException e) {
      response = makeFaultDocument(BAD_INTEGER_FORMAT, e.getMessage());
    }
    catch (IndexOutOfBoundsException e) {
      response = makeFaultDocument(NON_POSITIVE_INDEX, e.getMessage());
    }
    catch (Exception e) {
      response = makeFaultDocument(UNEXPECTED_PROBLEM, e.getMessage());
    }

    // Transform onto the OutputStream
    try {
      Source input = new DOMSource(response);
      Result output = new StreamResult(out);
      idTransform.transform(input, output);
      servletResponse.flushBuffer();
      out.flush();
      out.println();
    }
    catch (TransformerException e) {
      // If we get an exception at this point, it's too late to
      // switch over to an XML-RPC fault.
      throw new ServletException(e);
    }

  }

  // Given a node which does not contain any Element children,
  // accumulate all its text content from both text nodes and
  // CDATA sections (but not comments or processing instructions
  // and return it as a single string.
  private static String getFullText(Node node) {

    StringBuffer result = new StringBuffer();

    NodeList children = node.getChildNodes();
    for (int i = 0; i < children.getLength(); i++) {
      Node child = children.item(i);
      int type = child.getNodeType();
      if (type == Node.TEXT_NODE
       || type == Node.CDATA_SECTION_NODE) {
        result.append(child.getNodeValue());
      }
      else if (type == Node.ENTITY_REFERENCE_NODE) {
        // The JAXP spec is unclear about whether or not it's
        // possible for entity reference nodes to appear in the
        // tree. Just in case, let's expand them recursively:
        result.append(getFullText(child));
        // Validity does require that if they do appear their
        // replacement text is pure text, no elements.
      }
    }

    return result.toString();

  }

  // If performance is an issue, this could be pre-built in the
  // init() method and then cached. You'd just change one text
  // node each time.  This would only work in a SingleThreadModel
  // servlet.
  public Document makeResponseDocument(BigInteger result) {

    Document response
     = impl.createDocument(null, "methodResponse", null);

    Element methodResponse = response.getDocumentElement();
    Element params         = response.createElement("params");
    Element param          = response.createElement("param");
    Element value          = response.createElement("value");
    Element doubleElement  = response.createElement("double");
    Text    text = response.createTextNode(result.toString());

    methodResponse.appendChild(params);
    params.appendChild(param);
    param.appendChild(value);
    value.appendChild(doubleElement);
    doubleElement.appendChild(text);

    return response;

  }

  public Document makeFaultDocument(int faultCode, String faultString) {

    Document faultDoc
     = impl.createDocument(null, "methodResponse", null);

    Element methodResponse = faultDoc.getDocumentElement();

    Element fault         = faultDoc.createElement("fault");
    Element value         = faultDoc.createElement("value");
    Element struct        = faultDoc.createElement("struct");
    Element memberCode    = faultDoc.createElement("member");
    Element nameCode      = faultDoc.createElement("name");
    Text    nameCodeText  = faultDoc.createTextNode("faultCode");
    Element valueCode     = faultDoc.createElement("value");
    Element intCode       = faultDoc.createElement("int");
    String  codeString    = String.valueOf(faultCode);
    Text    textCode      = faultDoc.createTextNode(codeString);
    Element doubleElement = faultDoc.createElement("double");
    Element memberString  = faultDoc.createElement("member");
    Element nameString    = faultDoc.createElement("name");
    Text    nameText    = faultDoc.createTextNode("faultString");
    Element valueString   = faultDoc.createElement("value");
    Element stringString  = faultDoc.createElement("string");
    Text    textString    = faultDoc.createTextNode(faultString);

    methodResponse.appendChild(fault);
    fault.appendChild(value);
    value.appendChild(struct);
    struct.appendChild(memberCode);
    struct.appendChild(memberString);
    memberCode.appendChild(nameCode);
    memberCode.appendChild(valueCode);
    memberString.appendChild(nameString);
    memberString.appendChild(valueString);
    nameCode.appendChild(nameCodeText);
    nameString.appendChild(nameText);
    valueCode.appendChild(intCode);
    valueString.appendChild(stringString);
    intCode.appendChild(textCode);
    stringString.appendChild(textString);

    return faultDoc;

  }

  public static BigInteger calculateFibonacci(int generations)
   throws IndexOutOfBoundsException {

    if (generations < 1) {
      throw new IndexOutOfBoundsException(
       "Fibonacci numbers are not defined for " + generations
       + "or any other number less than one.");
    }
    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;
    for (int i = 2; i <= generations; i++) {
      BigInteger temp = high;
      high = high.add(low);
      low = temp;
    }
    return low;

  }

}

To compile FibonacciXMLRPCDOMServlet, you'll need to install the Java Servlet API somewhere in your class path. This is not included in the default distribution of the JDK. To get this to run in most servlet containers, you’ll need to add the JAR files for the DOM and JAXP implementations to the servlet’s library. In Tomcat, that directory is $TOMCAT_HOME/lib. It is not enough to have them in the virtual machine’s default ext directory because most servlet engines do not load classes from there. Details are in Chapter 3.

The servlet is divided into six methods. The init() method, the servlet substitute for a constructor, is responsible for finding a DOMImplementation class and loading a parser and a transformer engine. There’s no reason to waste time creating new ones for each request.

doPost() is the standard servlet method for responding to HTTP POST. Each POST to this servlet represents a separate XML-RPC request. This method first uses getElementsByTagName() to find the single int element in this request. Then it extracts the text content of this element and converts it to a Java int. This is more involved than you might expect because it’s necessary to consider the possibility that this text might not be part of a single node. For example, any of these three legal elements would give the int element multiple children:

<int>12<!-- Why is this comment here? -->34</int>
<int><?target data?>1234</int
<int>12<![CDATA[34]]></int>

Admittedly these are edge cases, but they have to be handled because they are legal. Comments and CDATA sections could be eliminated at parse time with the DocumentBuilderFactory’s setCoalescing(true) and setIgnoringComments(true) methods. However, that still leaves the possibility of a processing instruction.

There are also a lot of illegal things we aren’t handling. For instance, nothing notices if the methodName element is missing or if the root element isn’t methodCall. The proper way to handle this is to write a schema and validate the document against it before processing. An XML-RPC schema was demonstrated in Example 2.14. However, actually validating against this requires resorting to parser-specific classes. If you’re using Xerces, the following code would do the trick:

DOMParser parser = new DOMParser();
parser.setErrorHandler(YourErrorHandler);
parser.setFeature(
 "http://apache.org/xml/features/validation/schema", true);
parser.setProperty(
 "http://apache.org/xml/properties/schema/"
 + "external-noNamespaceSchemaLocation",
 "http://example.com/schemas/xmlrpc.xsd");
parser.parse(in);
Document doc = parser.getDocument();
// Work with the parser as before…

YourErrorHandler is an instance of some org.xml.sax.ErrorHandler implementation that throws a SAXException on detecting a validity error. Other schema-validating parsers like Oracle have slightly different APIs for checking a document against a known schema. Neither JAXP nor DOM2 provides a standard way to do this. When finished and implemented, DOM3 should allow you to perform schema-validation in a parser independent fashion.

Assuming the request document is valid, the next step is to calculate the requested Fibonacci number. The calculateFibonacci() method does this. There’s nothing new here, just math. This method doesn’t have to know anything about XML, DOM, or XML-RPC. If the original XML-RPC request contained a non-positive integer, then this method detects it and throws an IndexOutOfBoundsException. This will be caught in the doPost() method and converted into an XML-RPC fault response.

Once the result has been calculated, the makeResponseDocument() method wraps it up in properly formatted XML-RPC. If at any point something goes wrong, e.g. the request document is missing the required int element, an exception is thrown. This is caught and instead of the normal response, makeFaultDocument() is called to produce a proper XML-RPC fault response document.

Finally, a JAXP identity transform copies the finished result document onto the servlet’s output Writer. There’s not a lot we can do about an exception at this point, so any TransformerExceptions caught during the transform are converted into a ServletException with the original exception being available as the root cause of the ServletException. We can’t just let the TransformerException exception bubble up because doPost() in the superclass is not declared to throw it.

The SOAP servlet is similar in structure. However, namespaces are significant in SOAP so getElementsByTagNameNS() and other namespace-aware methods should be used. Example 10.14 demonstrates.

Example 10.14. A DOM based SOAP servlet

import javax.servlet.*;
import javax.servlet.http.*;
import java.io.*;
import java.math.BigInteger;
import org.w3c.dom.*;
import org.xml.sax.SAXException;
import javax.xml.parsers.*;
import javax.xml.transform.*;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;

public class FibonacciSOAPDOMServlet extends HttpServlet
  implements SingleThreadModel {

  // Fault codes
  public final static String MALFORMED_REQUEST_DOCUMENT
   = "MalformedRequest";
  public final static String INVALID_REQUEST_DOCUMENT
   = "InvalidRequest";
  public final static String INDEX_MISSING
   = "IndexMissing";
  public final static String NON_POSITIVE_INDEX
   = "NonPositiveIndex";
  public final static String BAD_INTEGER_FORMAT
  = "BadIntegerFormat";
  public final static String UNEXPECTED_PROBLEM
  = "UnexpectedProblem";

  private DocumentBuilder   parser;
  private DOMImplementation impl;
  private Transformer       idTransform;

  // Load a parser, transformer, and implementation
  public void init() throws ServletException {

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      // Always turn on namespace awareness
      factory.setNamespaceAware(true);

      this.parser = factory.newDocumentBuilder();
      this.impl   = parser.getDOMImplementation();

    }
    catch (Throwable t) {
      throw new ServletException(
       "Could not locate a JAXP parser", t);
    }

    try {
      TransformerFactory xformFactory = TransformerFactory.newInstance();
      this.idTransform = xformFactory.newTransformer();
    }
    catch (Throwable t) {
      throw new ServletException(
       "Could not locate a JAXP transformer", t);
    }

  }

  public void doPost(HttpServletRequest servletRequest,
   HttpServletResponse servletResponse)
   throws ServletException, IOException {

    servletResponse.setContentType("text/xml; charset=UTF-8");
    PrintWriter out = servletResponse.getWriter();
    InputStream in  = servletRequest.getInputStream();

    Document request;
    Document response;
    String generations ="here";
    try {
      request = parser.parse(in);

      NodeList ints = request.getElementsByTagNameNS(
       "http://namespaces.cafeconleche.org/xmljava/ch3/",
       "calculateFibonacci");
      Node input = ints.item(0);
      generations = getFullText(input);
      int numberOfGenerations = Integer.parseInt(generations);
      BigInteger result = calculateFibonacci(numberOfGenerations);
      response = makeResponseDocument(result);
    }
    catch (SAXException e) {
      response = makeFaultDocument(MALFORMED_REQUEST_DOCUMENT,
       e.getMessage());
    }
    catch (NullPointerException e) {
      response = makeFaultDocument(INDEX_MISSING,
       e.getMessage());
    }
    catch (NumberFormatException e) {
      response = makeFaultDocument(BAD_INTEGER_FORMAT,
       generations + e.getMessage());
    }
    catch (IndexOutOfBoundsException e) {
      response = makeFaultDocument(NON_POSITIVE_INDEX,
       e.getMessage());
    }
    catch (Exception e) {
      response = makeFaultDocument(UNEXPECTED_PROBLEM,
       e.getMessage());
    }

    // Transform onto the OutputStream
    try {
      Source input = new DOMSource(response);
      Result output = new StreamResult(out);
      idTransform.transform(input, output);
      servletResponse.flushBuffer();
      out.flush();

      out.println();
    }
    catch (TransformerException e) {
      // If we get an exception at this point, it's too late to
      // switch over to a SOAP fault
      throw new ServletException(e);
    }

  }

  private static String getFullText(Node node) {

    StringBuffer result = new StringBuffer();

    NodeList children = node.getChildNodes();
    for (int i = 0; i < children.getLength(); i++) {
      Node child = children.item(i);
      int type = child.getNodeType();
      if (type == Node.TEXT_NODE
       || type == Node.CDATA_SECTION_NODE) {
        result.append(child.getNodeValue());
      }
    }

    return result.toString();

  }

  // The details of the formats and namespace URIs are likely to
  // change when SOAP 1.2 is released.
  public Document makeResponseDocument(BigInteger result) {

    Document response
     = impl.createDocument(
      "http://schemas.xmlsoap.org/soap/envelope/",
      "SOAP-ENV:Envelope", null);

    Element envelope = response.getDocumentElement();
    Element body = response.createElementNS(
     "http://schemas.xmlsoap.org/soap/envelope/",
     "SOAP-ENV:Body");
    envelope.appendChild(body);

    Element Fibonacci_Numbers = response.createElementNS(
     "http://namespaces.cafeconleche.org/xmljava/ch3/",
     "Fibonacci_Numbers");
    body.appendChild(Fibonacci_Numbers);

    Element fibonacci = response.createElementNS(
     "http://namespaces.cafeconleche.org/xmljava/ch3/",
     "fibonacci");
    Fibonacci_Numbers.appendChild(fibonacci);

    Text text = response.createTextNode(result.toString());
    fibonacci.appendChild(text);

    return response;

  }

  public Document makeFaultDocument(String code, String message){

    Document faultDoc = impl.createDocument(
      "http://schemas.xmlsoap.org/soap/envelope/",
      "SOAP-ENV:Envelope", null);

    Element envelope = faultDoc.getDocumentElement();

    Element body = faultDoc.createElementNS(
     "http://schemas.xmlsoap.org/soap/envelope/",
     "SOAP-ENV:Body");
    envelope.appendChild(body);

    Element fault = faultDoc.createElementNS(
     "http://schemas.xmlsoap.org/soap/envelope/", "Fault");
    body.appendChild(fault);

    Element faultCode = faultDoc.createElement("faultcode");
    fault.appendChild(faultCode);

    Element faultString = faultDoc.createElement("faultstring");
    fault.appendChild(faultString);

    Text textCode = faultDoc.createTextNode(code);
    faultCode.appendChild(textCode);

    Text textString = faultDoc.createTextNode(message);
    faultString.appendChild(textString);

    return faultDoc;

  }

  public static BigInteger calculateFibonacci(int generations)
   throws IndexOutOfBoundsException {

    if (generations < 1) {
      throw new IndexOutOfBoundsException(
       "Fibonacci numbers are not defined for " + generations
       + "or any other number less than one.");
    }
    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;
    for (int i = 2; i <= generations; i++) {
      BigInteger temp = high;
      high = high.add(low);
      low = temp;
    }
    return low;

  }

}

This example has the same basic structure as the XML-RPC version. That is, the init() method loads the parser, DOM implementation, and identity transform. The doPost() method reads the request data and delegates building the request to the makeResponseDocument() method. If anything goes wrong, the makeFaultDocument() is called to produce a properly formatted SOAP fault response. Finally, a JAXP ID transform serializes the response onto the network stream. The formats of the request and response are a little different, but the program flow is the same.

I did structure the building of the two response documents (success and fault) a little differently. FibonacciXMLRPCServlet built all the nodes first and then connected them to each other. Here I add them to the tree as soon as they’re created. There’s not a lot of reason to do it one way or the other. Just use whichever seems more natural to you.

The details can be a little opaque. In a real-world program I’d definitely add some comments showing an example of the documents each method builds. If you’re building a lot of XML-RPC or SOAP documents with varying parameters, then it wouldn’t hurt to either develop a more generic library or buy or borrow a third-part library such as Apache SOAP. Behind the scenes, they’re doing something very much like what I’ve done in these two examples.

Transferring nodes between documents

In DOM, every node belongs to exactly one document at all times. It cannot exist independently of a Document, and it cannot be part of more than one Document at the same time. This is why the Document interface serves as the factory for creating node objects of various kinds. Furthermore, in DOM2, a node cannot be detached from its original document and placed in a new document. However, this restriction is loosened somewhat in DOM3.

Copying Nodes

The importNode() method makes a copy of a node found in another document. The copy can then be inserted in the importing document’s tree using the usual methods like insertBefore() and appendChild(). The document from which the node is imported is not changed in any way.

public Node importNode(Node toBeImported, boolean deep)
    throws DOMException;

The deep argument determines whether or not all the node’s descendants are copied with it. If true, they are. If false, they aren’t.

Document and document type nodes cannot be imported. Trying to import one will throw a DOMException. Entity and notation nodes can be imported but cannot be added as children of the importing document’s document type node. Thus there’s really not a lot of reason to import them.

Most of the other kinds of nodes can be imported with pretty much the results you’d expect. (Just remember to pass true for the second argument. This is almost always what you want.) The only really tricky ones are entity reference nodes. Even if deep is true, the children of the entity reference node are not copied. Instead the replacement text (node list) of the imported entity reference depends on what the importing document’s DTD says it is, not the original replacement text.

Moving Nodes

DOM Level 3 adds an adoptNode() method that moves a node from one document to another. That is, the node is deleted from the original document and inserted into the new document:

public Node adoptNode(Node adoptee)
    throws DOMException;

Adopted nodes are always moved with all their descendants intact. Otherwise, this behaves pretty much like importNode. That is, document nodes, document type nodes, notation nodes, and entity nodes cannot be adopted. All other kinds can be. Descendants of entity reference nodes are deleted from the original document but not copied into the new document.

As usual with DOM3 methods, this is past the bleeding edge. The latest versions of Xerces support it, but no other parsers do. There are also still a lot of unresolved issues concerning the behavior of this method, so I wouldn’t rely on it. For the time being, its better to use a two-step procedure in which a node is first copied into the new document with importNode() and then deleted from the original document with removeChild().

Prev | Up | Next
The Document Interface as an Abstract Factory | Home | Normalization
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 27, 2002


Normalization
Prev Chapter 10. Creating XML Documents with DOM Next

Normalization

Implementations have quite a bit of leeway in exactly how they parse and serialize any given document. For example, a parser may represent CDATA sections as CDATASection objects or it may merge them into neighboring Text objects. A parser may include entity reference nodes in the tree, or it may instead include the nodes corresponding to each entity’s replacement text. A parser may include comments or it may ignore them. DOM Level 3 adds four methods to the Document interface to control exactly how a parser makes these choices.

public void normalizeDocument();
public boolean canSetNormalizationFeature(String name, boolean state);
public void setNormalizationFeature(String name, boolean state);
public boolean getNormalizationFeature(String name);

The canSetNormalizationFeature() method tests whether the implementation supports the desired value (true or false) for the named feature. The setNormalizationFeature() method sets the value of the named feature. It throws a DOMException with the error code NOT_FOUND_ERR if the implementation does not support the feature at all. It throws a DOMException with the error code NOT_SUPPORTED_ERR if the implementation does not support the requested value for the feature. (e.g. you try to set to true a feature that must have the value false). Finally, after all the features have been set, client code can invoke the normalizeDocument() method to modify the tree in accordance with the current values for all the different features.

Caution

These are very bleeding edge ideas from the latest DOM Level 3 Core Working Draft. Xerces 2.0.2 is the only DOM implementation that supports any of this yet.

The DOM 3 specification defines 13 standard features:

normalize-characters, optional, default false

If true, document text should be normalized according to the W3C Character Model. For example, the word resumé would be represented as the six character string r e s u m é rather than the seven character string r e s u m e combining_'. Implementations are only required to support a false value for this feature.

split-cdata-sections, required, default true

If true, CDATA sections containing the CDATA section end delimiter ]]> are split into pieces and the ]]> included in a raw text node. If false, such a CDATA section is not split.

entities, optional, default true

If false, entity reference nodes are replaced by their children. If true, they’re not.

whitespace-in-element-content, optional, default true

If true, all white space is retained. If false, text nodes containing only white space are deleted if the parent element’s declaration from the DTD/schema does not allow #PCDATA to appear at that point.

discard-default-content, required, default true

If true, the implementation will throw away any nodes whose presence can be inferred from the DTD or schema; e.g. default attribute values.

canonical-form, optional, default false

If true, the document will be arranged according to the rules specified by the Canonical XML specification, at least within the limits of what can be represented in a DOM implementation. For example, EntityReference nodes would be replaced by their content and CDATASection objects would be converted to Text objects. However, there’s no way in DOM to specify everything canonicalization requires. For instance, a DOM Element does not know the order of its attributes or whether an empty element will be written as a single empty-element tag or start-tag/end-tag pair. Thus, full canonicalization has to be deferred to serialization time.

namespace-declarations, optional, default true

If false, then all Attr nodes representing namespace declarations are deleted from the tree. otherwise they’re retained. This has no effect on the namespaces associated with individual elements and attributes.

validate, optional, default false

If true, then the document’s schema or DTD is used to validate the document as it is being normalized. Any validation errors that are discovered are reported to the registered error handler. (Both validation and error handlers are other new features in DOM3.)

validate-if-schema, optional, default false

If true and the validation feature is also true, then the document is validated if and only if it has a some kind of schema (DTD, W3C XML Schema Language schema, RELAX NG schema, etc.).

datatype-normalization, required, default false

If true, datatype normalization is performed according to the schema. For example, an element declared to have type xsd:boolean and represented as <state>1</state> could be changed to <state>true</state>.

cdata-sections, optional, default true

If false, all CDATASection objects are changed into Text objects and merged with any adjacent Text objects. If true, each CDATA section is represented as its own CDATASection object.

comments, required, default true

If true, comments are included in the Document; if false, they’re not.

infoset, optional

If true the Document only contains information provided by the XML Infoset. This is the same as setting namespace-declarations, validate-if-schema, entities, and cdata-sections to false and datatype-normalization, whitespace-in-element-content, and comments to true.

In addition, vendors are allowed to define their own non-standard features. Feature names must be XML 1.0 names, and should use a vendor specific prefix such as apache: or oracle: .

For an example of how these could be useful, consider the SOAP servlet earlier in this chapter. It needed to locate the calculateFibonacci element in the request document and extract its full text content. This had to work even if that element contained comments and CDATA sections. The getFullText() method that accomplished this wasn’t too hard to write. Nonetheless, in DOM3 it’s even easier. Set the create-cdata-nodes and comments features to false and call normalizeDocument() as soon as the document is parsed. Once this is done, the calculateFibonacci element only contains one text node child.

try {
    Document request = parser.parse(in);
    request.setNormalizationFeature("create-cdata-nodes", false);
    request.setNormalizationFeature("comments", false);
    request.normalizeDocument();

    NodeList ints = request.getElementsByTagNameNS(
       "http://namespaces.cafeconleche.org/xmljava/ch3/",
       "calculateFibonacci");
    Node calculateFibonacci = ints.item(0);
    Node text = calculateFibonacci.getFirstChild();
    String generations = text.getNodeValue();
    // ...
}
catch (DOMException e) {
  // The create-cdata-nodes features is true by default and
  // parsers aren’t required to support a false value for it, so
  // you should be prepared to fall back on manual normalization
  // if necessary. The comments feature, however, is required.
}

This wouldn’t work for the XML-RPC case, however, because XML-RPC documents can contain processing instructions, and there’s no feature to turn off processing instructions.

Prev | Up | Next
The Document Interface as a Node Type | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 29, 2002


Summary
Prev Chapter 10. Creating XML Documents with DOM Next

Summary

DOM is designed around the abstract factory design pattern. The DOMImplementation interface is the abstract factory that creates implementation-specific instances of the Document and DocumentType interfaces. A vendor specific DOMImplementation object can be retrieved either by JAXP, by vendor-specific classes, or by the DOMImplementationRegistry class in DOM Level 3.

The Document interface is in turn an abstract factory that creates implementation-specific instances of Element, Text, Comment, Attr, ProcessingInstruction, EntityReference, CDATASection, and DocumentFragment. Each instance of these classes can only be part of the Document that created it. However, the Document interface also has an importNode() method that copies a node from one document to another.

The Document interface is also DOM’s representation of the root node of the tree. It provides methods to get the implementation that created the Document object, get the root element of the document, get the DOCTYPE of the document, import nodes into the document from other documents, and find elements by name, namespace, and ID. This last capability is often enough for many simple search applications.

Finally, don’t forget that Document is a sub-interface of Node and has all the methods of that interface that were discussed in the last chapter. This is crucial for adding content to and removing content from documents, especially at the top level outside the root element.

Prev | Up | Next
Normalization | Home | Chapter 11. The Document Object Model Core
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified December 31, 2001


Chapter 11. The Document Object Model Core
Prev Next

Chapter 11. The Document Object Model Core

Table of Contents

The Element Interface
Extracting Elements
Attributes
The NamedNodeMap Interface
The CharacterData interface
The Text Interface
The CDATASection Interface
The EntityReference Interface
The Attr Interface
The ProcessingInstruction Interface
The Comment Interface
The DocumentType Interface
The Entity Interface
The Notation Interface
Summary

The last two chapters have considered a DOM document to be mostly a tree of nodes. That is, it is composed of instances of the Node interface; and indeed for many purposes this is all you need to know. However, not all nodes are the same. Elements have properties that attributes don’t have. Attributes have properties that processing instructions don’t have. Processing instructions have properties comments don’t have, and so forth. In this chapter, we look at the unique properties and methods of the individual interfaces that make up an XML document.

The Element Interface

The Element interface is perhaps the most important of all the DOM component interfaces. After all, XML documents can be written without any comments, processing instructions, attributes, CDATA sections, entity references, or even text nodes. However, every XML document has at least one element and most have many more. Elements, more than any other component, define the structure of an XML document.

Example 11.1 summarizes the Element interface. This interface includes methods to get the prefixed name of the element, manipulate the attributes on the element, and select from the element’s descendants. Of course, Element objects also have all the methods of the Node super-interface such as appendChild() and getNamespaceURI().

Example 11.1. The Element interface

package org.w3c.dom;

public interface Element extends Node {

  public String  getTagName();

  public boolean hasAttribute(String name);
  public boolean hasAttributeNS(String namespaceURI,
   String localName);
  public String  getAttribute(String name);
  public void    setAttribute(String name, String value)
   throws DOMException;
  public void    removeAttribute(String name)
   throws DOMException;
  public Attr    getAttributeNode(String name);
  public Attr    setAttributeNode(Attr newAttr)
   throws DOMException;
  public Attr    removeAttributeNode(Attr oldAttr)
   throws DOMException;
  public String  getAttributeNS(String namespaceURI,
   String localName);
  public void    setAttributeNS(String namespaceURI,
   String qualifiedName, String value) throws DOMException;
  public void    removeAttributeNS(String namespaceURI,
   String localName) throws DOMException;
  public Attr    getAttributeNodeNS(String namespaceURI,
   String localName);
  public Attr    setAttributeNodeNS(Attr newAttr)
   throws DOMException;

  public NodeList getElementsByTagName(String name);
  public NodeList getElementsByTagNameNS(String namespaceURI,
   String localName);

}

The aesthetics of this interface are seriously marred by DOM’s requirement to avoid method overloading. The differences in the argument lists are redundantly repeated in the method names. For instance, if DOM had been written in pure Java, there’d probably be three setAttribute() methods with these signatures:

public void setAttribute(String name, String value)
    throws DOMException;
public void setAttribute(String namespaceURI, String localName, String value)
    throws DOMException;
public void setAttribute(Attr attribute)
    throws DOMException;

Instead, Element has these four methods with slightly varying names:

public void setAttribute(String name, String value)
    throws DOMException;
public void setAttributeNS(String namespaceURI, String localName, String value)
    throws DOMException;
public void setAttributeNode(Attr attribute)
    throws DOMException;
public void setAttributeNodeNS(Attr attribute)
    throws DOMException;

The distinction between setAttributeNode() and setAttributeNodeNS is unnecessary. setAttributeNode() can only be used for attributes in no namespace, whereas setAttributeNodeNS() can only be used with attributes in a namespace. The only motivation I can imagine for this is symmetry with the getter methods, where the distinction is relevant because the argument lists are different. For the setter methods though, this is frankly a mistake. Attr objects include their own namespace information. There’s no need for separate methods to set nodes with and without namespaces.

Extracting Elements

The getElementsByTagName() and getElementsByTagNameNS() methods behave the same as the similarly named methods in Document that you encountered in the last chapter. The only difference is that they search through a single element instead of the entire document. These methods return a NodeList containing all the elements with the specified name.

An asterisk (*) can be passed as either argument to indicate that all names or namespaces are desired. This is particularly useful for the local name passed to getElementsByTagNameNS(). For example, this NodeList would contain all RDF elements that are descendants of element:

NodeList rdfs = element.getElementsByTagNameNS(
 "http://www.w3.org/1999/02/22-rdf-syntax-ns#", "*");

The list returned is sorted in document order. In other words, elements are arranged in order of the appearance of their start-tags. If the start-tag for element A appears earlier in the document than the start-tag for element B, then element A comes before element B in the list.

The next example was inspired by the source code for this very book. Before publishing, I had to extract all the code examples from the source text and put them in separate files. The examples from each chapter go into separate directories. That is, the examples from Chapter 1 go into examples/1; the examples from Chapter 2 go into examples/2; and so forth. XSLT 1.0 isn’t quite up to this task, but DOM and Java are more powerful.[1]

The source code for this book is structured like this:

<book>
  …
  <chapter>
    …
    <example id="filename.java">
      <title>Some Java Program</title>
      <programlisting>import javax.xml.parsers;
        // more Java code…
      </programlisting>
    </example>
    …
    <example id="filename.xml">
      <title>Some XML document</title>
      <programlisting><![CDATA[<?xml version="1.0"?>
<root>
  …
</root>]]></programlisting>
     </example>
    …
  </chapter>
  more chapters…
</book>

At least, that’s the part that’s relevant to this example. The advantage to getElementsByTagName() and getElementsByTagNameNS() is that a program can extract just the parts that interest it very straightforwardly without explicitly walking the entire tree.[2] These methods effectively flatten the hierarchy to just the elements of interest. In this case those elements are chapter and example. Inside each example, the complete structure is somewhat more relevant so the normal tree-walking methods of Node are indicated.

The program follows these steps:

  1. Parse the entire book into a Document object.

  2. Use Document’s getElementsByTagName() method to retrieve a list of all the chapter elements in the document. (DocBook doesn’t use namespaces so getElementsByTagName() is chosen over getElementsByTagNameNS().)

  3. For each element in that list, use Element’s getElementsByTagName() method to retrieve a list of all the example elements in that chapter.

  4. From each element in that list, extract its programlisting child element.

  5. Write the text content of that programlisting element into a new file named by the ID of the example element

This example is quite specific to one XML application, DocBook. Indeed it won’t even work with all DocBook documents because it relies on various private conventions of this particular DocBook document, especially that the id attribute of each example element contains a file name. However, that’s OK. Most programs you write will be designed to process only certain XML documents in certain situations.

To increase robustness, I do require that the DocBook document be valid, and the parser does validate the document. If validation fails, this program aborts without extracting the examples, since it can’t be sure whether the document meets the preconditions. Example 11.2 demonstrates.

Example 11.2. Extracting examples from DocBook

import javax.xml.parsers.*;
import org.w3c.dom.*;
import org.xml.sax.*;
import java.io.*;

public class ExampleExtractor {

  public static void extract(Document doc) throws IOException {

    NodeList chapters = doc.getElementsByTagName("chapter");

    for (int i = 0; i < chapters.getLength(); i++) {

      Element chapter = (Element) chapters.item(i);
      NodeList examples = chapter.getElementsByTagName("example");

      for (int j = 0; j < examples.getLength(); j++) {

        Element example = (Element) examples.item(j);
        String fileName = example.getAttribute("id");
        // All examples should have id attributes but it's safer
        // not to assume that
        if (fileName == null) {
          throw
           new IllegalArgumentException("Missing id on example");
        }
        NodeList programlistings
         = example.getElementsByTagName("programlisting");
        // Each example is supposed to contain exactly one
        // programlisting, but we should verify that
        if (programlistings.getLength() != 1) {
          throw new
           IllegalArgumentException("Missing programlisting");
        }
        Element programlisting = (Element) programlistings.item(0);

        // Extract text content; this is a little tricky because
        // these often contain CDATA sections and entity
        // references which can be represented as separate nodes
        // so we can't just ask for the first text node child of
        // each program listing.
        String code = getText(programlisting);

        // write code into a file
        File dir = new File("examples2/" + i);
        dir.mkdirs();
        File file = new File(dir, fileName);
        System.out.println(file);
        FileOutputStream fout = new FileOutputStream(file);
        Writer out = new OutputStreamWriter(fout, "UTF-8");
        // Buffering almost always helps performance a lot
        out = new BufferedWriter(out);
        out.write(code);
        // Be sure to remember to flush and close your streams
        out.flush();
        out.close();

      } // end examples loop

    } // end chapters loop

  }

  public static String getText(Node node) {

    // We need to retrieve the text from elements, entity
    // references, CDATA sections, and text nodes; but not
    // comments or processing instructions
    int type = node.getNodeType();
    if (type == Node.COMMENT_NODE
     || type == Node.PROCESSING_INSTRUCTION_NODE) {
       return "";
    }

    StringBuffer text = new StringBuffer();

    String value = node.getNodeValue();
    if (value != null) text.append(value);
    if (node.hasChildNodes()) {
      NodeList children = node.getChildNodes();
      for (int i = 0; i < children.getLength(); i++) {
        Node child = children.item(i);
        text.append(getText(child));
      }
    }

    return text.toString();

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java ExampleExtractor URL");
      return;
    }
    String url = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setValidating(true);

      DocumentBuilder parser = factory.newDocumentBuilder();
      parser.setErrorHandler(new ValidityRequired());

      // Read the document
     Document document = parser.parse(url);

     // Extract the examples
     extract(document);

    }
    catch (SAXException e) {
      System.out.println(e);
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not read " + url
      );
      System.out.println(e);
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  } // end main

}

// Make validity errors fatal
class ValidityRequired implements ErrorHandler {

  public void warning(SAXParseException e)
    throws SAXException {
    // ignore warnings
  }

  public void error(SAXParseException e)
   throws SAXException {
    // Mostly validity errors. Make them fatal.
    throw e;
  }

  public void fatalError(SAXParseException e)
   throws SAXException {
    throw e;
  }

}

Since ExampleExtractor is fairly involved, I’ve factored it into several relatively independent pieces. The main() method builds the document and parses the document as usual. The non-public class ValidityRequired more or less converts all errors into fatal errors by rethrowing the exception it’s been passed. Assuming validation succeeds, the document is then passed to the extract() method.

The extract() method iterates through all the chapters and examples in the book using getElementsByTagName(). Each example is supposed to have an id attribute and a single programlisting child element. However, since this is just a convention for this one document rather than a rule enforced by the DTD, the code checks to make sure that’s true. If it isn’t, it throws an IllegalArgumentException.

Next comes one of the trickiest part of working with elements in DOM. I need to extract the text content of the programlisting element, which sounds simple enough. However, there’s no method in either Element or Node that performs this routine task. You might expect getNodeValue() to do this, especially if you’re used to XPath. However, in DOM, unlike XPath, the value of an element is null. Only its children have values. Thus we need to recursively descend through the children of the programlisting element, accumulating the values of all text nodes, entity references, CDATA sections, and other elements as we go. The getText() method accomplishes this.

Once we’ve got the actual example code from the programlisting element, it can be written into a file. The file location is relative to the current working directory and the chapter number. The file name has been read from the id attribute. UTF-8 works well as the default encoding.

Attributes

Although DOM has an Attr interface, the Element interface is the primary means of reading and writing attributes. Since each element can have no more than one attribute with the same name, attributes can be stored and retrieved just by their names. There’s no need to manage complex list structures, as there is with other kinds of nodes.

Here are a few tips that explain how the attribute methods work in DOM:

  • Most attributes are not in any namespace. In particular, unprefixed attributes are never in any namespace. For these attributes, just use the name and value.

  • For attributes that are in a namespace, specify the prefixed name and URI when setting them. Specify the local name and namespace URI when getting them.

  • Getting the value of an attribute that doesn’t exist returns the empty string.

  • Setting an attribute that already exists changes the value of the existing attribute.

With these few principles in mind, it’s straightforward to write programs that read attributes. I'll demonstrate by revising last chapter’s Fibonacci program. That example just used elements. Now let's add a index attribute to each fibonacci element as shown in Example 11.3:

Example 11.3. A document that uses attributes

<?xml version="1.0"?>
<Fibonacci_Numbers>
  <fibonacci index="1">1</fibonacci>
  <fibonacci index="2">1</fibonacci>
  <fibonacci index="3">2</fibonacci>
  <fibonacci index="4">3</fibonacci>
  <fibonacci index="5">5</fibonacci>
  <fibonacci index="6">8</fibonacci>
  <fibonacci index="7">13</fibonacci>
  <fibonacci index="8">21</fibonacci>
  <fibonacci index="9">34</fibonacci>
  <fibonacci index="10">55</fibonacci>
</Fibonacci_Numbers>

This is really quite simple to implement. You just need to calculate a string name and value for the attribute and call setAttribute() in the right place. Example 11.4 demonstrates.

Example 11.4. A DOM program that adds attributes

import org.w3c.dom.*;
import javax.xml.parsers.*;
import javax.xml.transform.*;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import java.math.BigInteger;

public class FibonacciAttributeDOM {

  public static void main(String[] args) {

    try {

      // Find the implementation
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setNamespaceAware(true);
      DocumentBuilder builder = factory.newDocumentBuilder();
      DOMImplementation impl = builder.getDOMImplementation();

      // Create the document
      Document doc = impl.createDocument(null,
       "Fibonacci_Numbers", null);

      // Fill the document
      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      Element root = doc.getDocumentElement();

      for (int i = 0; i < 10; i++) {
        Element number = doc.createElement("fibonacci");
        String value = Integer.toString(i);
        number.setAttribute("index", value);
        Text text = doc.createTextNode(low.toString());
        number.appendChild(text);
        root.appendChild(number);

        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }

      // Serialize the document onto System.out
      TransformerFactory xformFactory
       = TransformerFactory.newInstance();
      Transformer idTransform = xformFactory.newTransformer();
      Source input = new DOMSource(doc);
      Result output = new StreamResult(System.out);
      idTransform.transform(input, output);

    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a JAXP factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println(
        "Could not locate a JAXP DocumentBuilder class"
      );
    }
    catch (DOMException e) {
      System.err.println(e);
    }
    catch (TransformerConfigurationException e) {
      System.err.println(e);
    }
    catch (TransformerException e) {
      System.err.println(e);
    }

  }

}

[1] XSLT 2.0 could handle this, and many XSLT engines include extension functions that could pull this off in XSLT 1.0. However, I needed the example. :-)

[2] A naive DOM implementation probably would implement getElementsByTagName() and getElementsByTagNameNS() by walking the tree or sub-tree, but more efficient implementations based on detailed knowledge of the data structures that implement the various interfaces also exist. For instance, a DOM that sits on top of a native XML database might have access to an index of all the elements in the document.

Prev | Up | Next
Summary | Home | The NamedNodeMap Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 10, 2002


The NamedNodeMap Interface
Prev Chapter 11. The Document Object Model Core Next

The NamedNodeMap Interface

If for some reason, you want all the attributes of an element or you don’t know their names, you can use the getAttributes() method to retrieve a NamedNodeMap inherited from the Node. (Why getAttributes() is in Node instead of Element I have no idea. Elements are the only kind of node that can have attributes. For all other types of node, getAttributes() returns null.) The NamedNodeMap interface, summarized in Example 11.5, has methods to get and set the various named nodes as well to iterate through the nodes like a list. Here it’s used for attributes, but soon you’ll see it used for notations and entities as well.

Example 11.5. The NamedNodeMap interface

package org.w3c.dom;

public interface NamedNodeMap {

  // for iterating through the map as a list
  public Node item(int index);
  public int  getLength();

  // For working with particular items in the list
  public Node getNamedItem(String name);
  public Node setNamedItem(Node arg) throws DOMException;
  public Node removeNamedItem(String name)
   throws DOMException;
  public Node getNamedItemNS(String namespaceURI,
   String localName);
  public Node setNamedItemNS(Node arg) throws DOMException;
  public Node removeNamedItemNS(String namespaceURI,
   String localName) throws DOMException;

}

I’ll demonstrate with an XLink spider program like the one you saw in Chapter 6. However, this time I’ll implement the program on top of DOM rather than SAX. You can judge for yourself which one is more natural.

Recall that XLink is an attribute based syntax for denoting connections between documents. The element that is the link has an xlink:type attribute with the value simple and an xlink:href attribute whose value is the URL of the remote document. For example, this book element points to this book’s home page:

<book xlink:type="simple"
      xlink:href="http://www.cafeconleche.org/books/xmljava/"
      xmlns:xlink="http://www.w3.org/1999/xlink">
  Processing XML with Java
</book>

The customary prefix xlink is bound to the namespace URI http://www.w3.org/1999/xlink. Most of the time you should depend on the specific URI and not the prefix, which may change.

Relative URLs are relative to the nearest ancestor xml:base attribute if one is present or the location of the document otherwise. For example, the book element in this library element also points to http://www.cafeconleche.org/books/xmljava/.

<library xml:base="http://www.cafeconleche.org/"
         xmlns:xlink="http://www.w3.org/1999/xlink">
  <book xlink:type="simple" xlink:href="books/xmljava/">
    Processing XML with Java
  </book>
</library>

The prefix xml is bound to the namespace URI http://www.w3.org/XML/1998/namespace. This is a special case, however. The xml prefix cannot be changed, and does not need to be declared.

Attributes provide all the information needed to process the link. Consequently, a spider can follow XLinks without knowing any details about the rest of the markup in the document. Example 11.6 is such a program. Currently this spider does nothing more than follow the links and print their URLs. However, it would not be hard to add code to load the discovered documents into a database or perform some other useful operation. You’d just subclass DOMSpider while overriding the process() method.

Example 11.6. An XLink spider that uses DOM

import org.xml.sax.SAXException;
import javax.xml.parsers.*;
import java.io.*;
import java.util.*;
import java.net.*;
import org.w3c.dom.*;

public class DOMSpider {

  public static String XLINK_NAMESPACE
   = "http://www.w3.org/1999/xlink";

  // This will be used to read all the documents. We could use
  // multiple parsers in parallel. However, it's a lot easier
  // to work in a single thread, and doing so puts some real
  // limits on how much bandwidth this program will eat.
  private DocumentBuilder parser;

  // Builds the parser
  public DOMSpider() throws ParserConfigurationException {

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setNamespaceAware(true);
      parser = factory.newDocumentBuilder();
    }
    catch (FactoryConfigurationError e) {
      // I don't absolutely need to catch this, but I hate to
      // throw an Error for no good reason.
      throw new ParserConfigurationException(
       "Could not locate a factory class");
    }

  }

  // store the URLs already visited
  private Vector visited = new Vector();

  // Limit the amount of bandwidth this program uses
  private int maxDepth = 5;
  private int currentDepth = 0;

  public void spider(String systemID) {
    currentDepth++;
    try {
      if (currentDepth < maxDepth) {
        Document document = parser.parse(systemID);
        process(document, systemID);

        Vector toBeVisited = new Vector();
        // search the document for uris,
        // store them in vector, and print them
        findLinks(document.getDocumentElement(),
         toBeVisited, systemID);

        Enumeration e = toBeVisited.elements();
        while (e.hasMoreElements()) {
          String uri = (String) e.nextElement();
          visited.add(uri);
          spider(uri);
        }

      }

    }
    catch (SAXException e) {
      // Couldn't load the document,
      // probably not well-formed XML, skip it
    }
    catch (IOException e) {
      // Couldn't load the document,
      // likely network failure, skip it
    }
    finally {
      currentDepth--;
      System.out.flush();
    }

  }

  public void process(Document document, String uri) {
    System.out.println(uri);
  }

  // Recursively descend the tree of one document
  private void findLinks(Element element, List uris,
   String base) {

    // Check for an xml:base attribute
    String baseAtt = element.getAttribute("xml:base");
    if (!baseAtt.equals(""))  base = baseAtt;

    // look for XLinks in this element
    if (isSimpleLink(element)) {
      String uri
       = element.getAttributeNS(XLINK_NAMESPACE, "href");
      if (!uri.equals("")) {
        try {
          String wholePage = absolutize(base, uri);
          if (!visited.contains(wholePage)
           && !uris.contains(wholePage)) {
            uris.add(wholePage);
          }
        }
        catch (MalformedURLException e) {
          // If it's not a good URL, then we can't spider it
          // anyway, so just drop it on the floor.
        }
      } // end if
    } // end if

    // process child elements recursively
    NodeList children = element.getChildNodes();
    for (int i = 0; i < children.getLength(); i++) {
      Node node = children.item(i);
      int type = node.getNodeType();
      if (type == Node.ELEMENT_NODE) {
        findLinks((Element) node, uris, base);
      }
    } // end for

  }

  // If you're willing to require Java 1.4, you can do better
  // than this with the new java.net.URI class
  private static String absolutize(String context, String uri)
   throws MalformedURLException {

    URL contextURL = new URL(context);
    URL url = new URL(contextURL, uri);
    // Remove fragment identifier if any
    String wholePage = url.toExternalForm();
    int fragmentSeparator = wholePage.indexOf('#');
    if (fragmentSeparator != -1) {
      // There is a fragment identifier
      wholePage = wholePage.substring(0, fragmentSeparator);
    }
    return wholePage;

  }

  private static boolean isSimpleLink(Element element) {

    String type
     = element.getAttributeNS(XLINK_NAMESPACE, "type");
    if (type.equals("simple")) return true;
    return false;

  }

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println("Usage: java DOMSpider topURL");
      return;
    }

    // start parsing...
    try {
      DOMSpider spider = new DOMSpider();
      spider.spider(args[0]);
    }
    catch (Exception e) {
      System.err.println(e);
      e.printStackTrace();
    }

  } // end main

} // end DOMSpider

There are two levels of recursion here. The spider() method recursively spiders documents. The findLinks() method recursively searches through the elements in a document looking for XLinks. It adds the URLs found in these links to a list of unvisited pages. After finishing each of these documents, the next document is retrieved from the list and processed in turn. If it’s an XML document, then it is parsed and passed to the process() method. Non-XML documents found at the end of XLinks are ignored.

I tested this program by pointing it at the Resource Directory Description Language specification, which is one of the few real-world documents I know of that uses XLinks. I was surprised to find out just how much XLinked XML there is out there in the world, though as of yet most of it is just more XML specifications. This must be what the Web felt like circa 1991. Here’s a sample of the more interesting output:

D:\books\XMLJAVA>java DOMSpider http://www.rddl.org/
http://www.rddl.org/
http://www.rddl.org/purposes
http://www.rddl.org/purposes/software
http://www.rddl.org/rddl.rdfs
http://www.rddl.org/rddl-integration.rxg
http://www.rddl.org/modules/rddl-1.rxm
…
http://www.w3.org/2001/XMLSchema
http://www.w3.org/2001/XMLSchema.xsd
http://www.examplotron.org
http://www.examplotron.org/compile.xsl
http://www.examplotron.org/examplotron.xsd
http://www.examplotron.org/0/1/
http://www.examplotron.org/0/2/
http://www.examplotron.org/0/3/
http://webns.net/rdfs/
http://www.w3.org/2000/01/rdf-schema
http://webns.net/rdfs/?format=rdf
http://webns.net/foaf/
http://xmlns.com/foaf/0.1/
http://webns.net/foaf/?format=rdf
http://webns.net/dc/
http://purl.org/dc/elements/1.1/
http://webns.net/dc/?format=rdf
http://openhealth.org/XSet
http://xsltunit.org/0/1/
http://xsltunit.org/0/1/xsltunit.xsl
http://xsltunit.org/0/1/tst_library.xsl
http://xsltunit.org/0/1/library.xml
http://xsltunit.org/0/1/library.xsl
http://venetica.com/venicebridgecontent/
http://www.venetica.com/VeniceBridgeContent
http://www.venetica.com/VeniceBridgeContent/VeniceBridgeContent40.xsd
http://www.venetica.com/VeniceBridgeContent/VeniceBridgeContent.biz
http://www.venetica.com/VeniceBridgeContent/rddl30.html
http://www.w3.org/TR/xhtml-basic
http://www.w3.org/TR/xml-infoset/
http://www.w3.org/TR/xhtml-modularization/
Prev | Up | Next
Chapter 11. The Document Object Model Core | Home | The CharacterData interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified February 26, 2002


The CharacterData interface
Prev Chapter 11. The Document Object Model Core Next

The CharacterData interface

The CharacterData interface is a generic super-interface for nodes that are composed mostly of text including:

  • Text

  • CDATASection

  • Comment

The CharacterData interface is almost never used directly, rather than as an instance of one of these three sub-interfaces. However, you almost always work with text, comment, and CDATA section nodes using the methods of the CharacterData interface.

Example 11.7 summarizes the CharacterData interface. This interface has methods that manipulate the text content of this node. As usual, it also inherits all the methods of its super-interface Node such as getParentNode() and getNodeValue().

Example 11.7. The CharacterData interface

package org.w3c.dom;

public interface CharacterData extends Node {

  public String getData() throws DOMException;
  public void   setData(String data) throws DOMException;
  public int    getLength();
  public String substringData(int offset, int length)
   throws DOMException;
  public void   appendData(String data) throws DOMException;
  public void   insertData(int offset, String data)
   throws DOMException;
  public void   deleteData(int offset, int length)
   throws DOMException;
  public void   replaceData(int offset, int length, String data)
   throws DOMException;

}

The getData() method returns a String containing the complete content of the node. Any escaped characters like &amp; or &#160; will be replaced by the actual characters they represent. The setData() method replaces the entire text content of the node. There’s no need to escape the string you pass to this method. If the document is written out to a file or a stream, the serialization code is responsible for escaping these characters. In-memory, the type of the object is enough to determine whether a less than sign is the start of a tag or just a less than sign.

There are also methods to read and write just parts of the text content. The offsets are all zero-based as in Java’s String class. For example, this code fragment deletes the first six characters from the CharacterData object text:

text.delete(0, 6);

Java’s String type is a very good match for DOM strings. Each char in a Java String is a single UTF-16 code point. That is, most Unicode characters are represented by exactly one Java char. However, characters with code points greater than 65,535 such as many musical symbols are represented by two chars each, one for each half of the surrogate pair representing the character in UTF-16. The getLength() method in this interface returns the number of UTF-16 code points, not the number of Unicode characters. This is also how the length() method in Java’s String class behaves.

On Usenet, jokes which some people are likely to find offensive are often obscured by rotating the ASCII character set 13 places. That is, the first letter of the alphabet, A, is transformed into the fourteenth letter of the alphabet, N. The second letter of the alphabet, B, is transformed into the fifteenth letter of the alphabet, O, and so forth through M which becomes Z. Then N is transformed into A, O into B, and so on through Z which becomes M. It’s not a particularly strong cipher, but it’s enough to prevent people from accidentally reading something they don’t want to read. It has the extra advantage of reversing itself. That is, running the cipher text through the ROT13 algorithm one more time restores the original text.

Example 11.8 is a simple program that obscures text nodes, comments, and CDATA sections by applying the ROT13 algorithm to them. The encoded documents are as well-formed and valid as the unencoded documents. Only the character data gets changed, not the markup. [3] This program can also decode documents that are already encoded.

Example 11.8. ROT13 encoder for XML documents

import javax.xml.parsers.*;
import javax.xml.transform.*;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.dom.DOMSource;
import org.w3c.dom.*;
import org.xml.sax.SAXException;
import java.io.IOException;

public class ROT13XML {

  // note use of recursion
  public static void encode(Node node) {

    if (node instanceof CharacterData) {
      CharacterData text = (CharacterData) node;
      String data = text.getData();
      text.setData(rot13(data));
    }

    // recurse the children
    if (node.hasChildNodes()) {
      NodeList children = node.getChildNodes();
      for (int i = 0; i < children.getLength(); i++) {
        encode(children.item(i));
      }
    }

  }

  public static String rot13(String s) {

    StringBuffer out = new StringBuffer(s.length());
    for (int i = 0; i < s.length(); i++) {
      int c = s.charAt(i);
      if (c >= 'A' && c <= 'M') out.append((char) (c+13));
      else if (c >= 'N' && c <= 'Z') out.append((char) (c-13));
      else if (c >= 'a' && c <= 'm') out.append((char) (c+13));
      else if (c >= 'n' && c <= 'z') out.append((char) (c-13));
      else out.append((char) c);
    }
    return out.toString();

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java ROT13XML URL");
      return;
    }

    String url = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      DocumentBuilder parser = factory.newDocumentBuilder();

      // Read the document
      Document document = parser.parse(url);

      // Modify the document
      ROT13XML.encode(document);

      // Write it out again
      TransformerFactory xformFactory
       = TransformerFactory.newInstance();
      Transformer idTransform = xformFactory.newTransformer();
      Source input = new DOMSource(document);
      Result output = new StreamResult(System.out);
      idTransform.transform(input, output);

    }
    catch (SAXException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
      "Due to an IOException, the parser could not encode " + url
      );
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }
    catch (TransformerConfigurationException e) {
      System.out.println("Could not locate a TrAX transformer");
    }
    catch (TransformerException e) {
      System.out.println("Could not transform");
    }

  } // end main

}

The encode() method recursively descends the tree applying the ROT13 algorithm to every CharacterData object it finds, whether a Comment, Text, or CDATASection. The algorithm itself is encapsulated in the rot13() method. Since both methods merely operate on their arguments but otherwise have no interaction with any state maintained in the class, I made them static. The main() method encodes a document at a URL typed on the command line, and then copies the result to System.out.

Here’s a joke encoded by this program. You’ll have to run the program if you want to find out what it says. :-)

D:\books\XMLJAVA>java ROT13XML joke.xml
<?xml version="1.0" encoding="utf-8"?><joke>
  Erchoyvpna srghf-jbefuvccref jnag gb tvir srghfrf rdhny be
  fhcrevbe evtugf bire jbzra'f obqvrf, rira vs vg guerngraf n
  jbzna'f culfvpny urnygu -- rira jura gur srghf qbrfa'g lrg unir
  n shyyl shapgvbavat uhzna oenva, be nal oenva ng nyy. Lbh unir
  gb fnl bar guvat -- Erchoyvpnaf gnxr pner bs gurve bja.
</joke>

[3] ROT13XML could also encode attribute values and processing instructions without affecting well-formedness or validity, but since DOM does not represent these nodes as instances of CharacterData, I leave this as an exercise for the reader.

Prev | Up | Next
The NamedNodeMap Interface | Home | The Text Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 04, 2002


The Text Interface
Prev Chapter 11. The Document Object Model Core Next

The Text Interface

The Text interface represents a text node. This can be a child of an element, an attribute, or an entity reference. When a document is built by a parser, each text node will contain the longest possible run of contiguous parsed character data from the document, and thus no text node will be adjacent to any other. However, documents built in memory may contain adjacent text nodes. Invoking the normalize() method in the Node interface on any ancestor of the text nodes will merge these together.

Example 11.9 summarizes the Text interface. Besides the methods like setData() and getNodeValue() that Text inherits from its super-interfaces, it has one new method that splits a Text object into two.

Example 11.9. The Text interface

package org.w3c.dom;

public interface Text extends CharacterData {

  public Text splitText(int offset) throws DOMException;

}

The splitText() method changes one text node into two by dividing its data at a specified offset. All characters after the split are cut out of the original node. A new text node is created and returned. Both are included in the tree. If the offset is less than zero or greater than the length of the data, splitText() throws a DOMException with the code for INDEX_SIZE_ERR.

The main reason to split a text node is so that you can move or delete part of some text, but not the entire node. It can also be used to insert a new node in the middle of a run of text. For example, suppose date is an Element object representing this element:

<date>2002-01-08</date>

Now suppose you want to change date to represent this element:

<date><year>2002</year><month>01</month><day>08</day></date>

This code will do it:

Document document = date.getOwnerDocument();
Text yearText = (Text) date.getFirstChild();
Text slash = yearText.splitText(4);
Text monthText = slash.splitText(1);
Text nextSlash = monthText.splitText(2);
Text dayText = nextSlash.splitText(1);

Element year = document.createElement("year");
Element month = document.createElement("month");
Element day = document.createElement("day");

date.removeChild(slash);
date.removeChild(monthText);
date.removeChild(yearText);
date.removeChild(nextSlash);
date.removeChild(dayText);

year.appendChild(yearText);
month.appendChild(monthText);
day.appendChild(dayText);
date.appendChild(year);
date.appendChild(month);
date.appendChild(day);

A lot of the time these operations can be more straightforwardly implemented through String methods.

Example 11.10 is a simple program that recursively descends a DOM tree and prints all text nodes on System.out. This has the effect of stripping out the markup while leaving all text inside the document intact:

Example 11.10. Printing the text nodes in an XML document

import javax.xml.parsers.*;
import org.w3c.dom.*;
import org.xml.sax.SAXException;
import java.io.IOException;

public class DOMTextExtractor {

  public void processNode(Node node) {

    if (node instanceof Text) {
      Text text = (Text) node;
      String data = text.getData();
      System.out.println(data);
    }

  }

  // note use of recursion
  public void followNode(Node node) {

    processNode(node);
    if (node.hasChildNodes()) {
      NodeList children = node.getChildNodes();
      for (int i = 0; i < children.getLength(); i++) {
        followNode(children.item(i));
      }
    }

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java DOMTextExtractor URL");
      return;
    }

    String url = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      DocumentBuilder parser = factory.newDocumentBuilder();
      // If expandEntityReferences isn't turned off, there
      //  won't be any entity reference nodes in the DOM tree
      factory.setExpandEntityReferences(false);

      // Read the document
      Document document = parser.parse(url);

      // Process the document
      DOMTextExtractor extractor = new DOMTextExtractor();
      extractor.followNode(document);

    }
    catch (SAXException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check " + url
      );
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  } // end main

}

Here’s the result of running the XML specification through this program:

D:\books\XMLJAVA>java DOMTextExtractor
 http://www.w3.org/TR/2000/REC-xml-20001006.xml

Extensible Markup Language (XML)

1.0 (Second Edition)

REC-xml-20001006

W3C Recommendation

6
October
2000
…

Notice that white space is included in text nodes and is significant. Text inside entity references is also found, one way or another. If the DOM parser is producing entity reference nodes, then the replacement text of the entity becomes children of the entity reference nodes. Otherwise, the replacement text of the entity is simply resolved into the surrounding text nodes.

Prev | Up | Next
The CharacterData interface | Home | The CDATASection Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 29, 2002


The CDATASection Interface
Prev Chapter 11. The Document Object Model Core Next

The CDATASection Interface

The CDATASection interface, shown in Example 11.11, is a subinterface of Text that specifically represents CDATA sections. It has no unique methods of its own. However, when a CDATASection is serialized into a file, the text of the node may be wrapped inside CDATA section markers so that characters like & and < do not need to be escaped as &amp; and &lt;.

Example 11.11. The CDATASection interface

package org.w3c.dom;

public interface CDATASection extends Text {

}

CDATA sections are convenient syntax sugar for documents that will sometimes be read or authored by human beings in source code form. The source code for this book uses them frequently for examples. Please don’t use CDATA sections for more than that. With the possible exception of editors, all programs that process XML documents should treat CDATA sections as identical to the same text with all the less than signs changed to &lt; and all the ampersands changed to &amp;. In particular, do not use CDATA sections as a sort of pseudo-element to hide HTML in your XML documents like this:

  <Product>
    <Name>Brass Ship's Bell</Name>
    <Quantity>1</Quantity>
    <Price currency="USD">144.95</Price >
    <Discount>.10</Discount>
    <![CDATA[<html><body>
      <b>Happy Father&rsquo;s Day to a great Dad!<P></b>

      <i>Love,<br>
      Sam and Beatrice<body></html>]]>
  </Product>

Instead, write well-formed HTML inside an appropriate element like this:

  <Product>
    <Name>Brass Ship's Bell</Name>
    <Quantity>1</Quantity>
    <Price currency="USD">144.95</Price >
    <Discount>.10</Discount>
    <GiftMessage><html><body>
      <p><b>Happy Father's Day to a great Dad!</b></p>

      <i>Love,<br />
      Sam and Beatrice</i></body></html>
    </GiftMessage>
  </Product>

This is much more flexible and much more robust. DOM parsers are not required to report CDATA sections, and other processes are even less likely to maintain them so you should not use CDATA sections as a substitute for elements.

The normalize() method in the Node interface does not combine CDATA sections with adjacent text nodes or other CDATA sections. Example 11.12 provides a static utility method that does do this. A Node is passed in as an argument. All CDATASection descendants of this node are converted to simple Text objects and then all adjacent Text objects are merged. The argument is modified in place. Thus the method returns void.

Example 11.12. Merging CDATA sections with text nodes

import org.w3c.dom.*;

public class CDATAUtility {

  // Recursively descend the tree converting all CDATA sections
  // to text nodes and merging them with adjacent text nodes.
  public static void superNormalize(Node parent) {

    // We'll need this to create new Text objects
    Document factory = parent.getOwnerDocument();

    Node current = parent.getFirstChild();
    while (current != null) {

      int type = current.getNodeType();
      if (type == Node.CDATA_SECTION_NODE) {
        // Convert CDATA section to a text node
        CDATASection cdata = (CDATASection) current;
        String data = cdata.getData();
        Text newNode = factory.createTextNode(data);
        parent.replaceChild(newNode, cdata);
        current = newNode;
      }

      // Recheck in case we changed type above
      type = current.getNodeType();
      if (type == Node.TEXT_NODE) {
        // If previous node is a text node, then append this
        // node's data to that node, and delete this node
        Node previous = current.getPreviousSibling();
        if (previous != null) {
          int previousType = previous.getNodeType();
          if (previousType == Node.TEXT_NODE) {
            Text previousText = (Text) previous;
            Text currentText = (Text) current;
            String data = currentText.getData();
            previousText.appendData(data);
            parent.removeChild(current);
            current = previous;
          }
        }
      } // end if
      else { // recurse
        superNormalize(current);
      }

      // increment node
      current = current.getNextSibling();

    } // end while

  }  // end superNormalize()

}

More than anything else superNormalize() is an exercise in navigating the DOM tree. It uses the Node methods getFirstChild(), getNextSibling(), and getPreviousChild() in a while loop instead of iterating through a NodeList in a for loop because it’s constantly changing the contents of the node list. Node lists are live but keeping the loop counter pointed at the right node as the list changes is tricky (not impossible certainly, just not as straightforward as the approach used here).

Prev | Up | Next
The Text Interface | Home | The EntityReference Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 29, 2002


The EntityReference Interface
Prev Chapter 11. The Document Object Model Core Next

The EntityReference Interface

The EntityReference interface represents a general entity reference such as &nbsp; or &copyright_notice;. (It is not used for the five predefined entity references &amp;, &lt;, &gt;, &apos;, and &quot;.)

Example 11.13 summarizes the EntityReference interface. You’ll notice it declares exactly zero methods of its own. All of its functionality is inherited from the Node super-interface. In an XML document, an entity reference is just a place holder for the text the entity reference will be replaced by. In a DOM tree, an EntityReference object merely contains the things the entity reference will be replaced by.

Example 11.13. The EntityReference interface

package org.w3c.dom;

public interface EntityReference extends Node {

}

The name of the entity reference is returned by the getNodeName() method. The replacement text for the entity (assuming the parser has resolved the entity) can be read through the usual methods of the Node interface. like getFirstChild(). However, entity references are read-only. You cannot change their children using methods like appendChild() or replaceChild() or change their names using methods like setNodeName(). Trying to do so throws a DOMException with the error code NO_MODIFICATION_ALLOWED_ERR.

EntityReference objects do not know their own system ID (URL) or public ID. However using the entity reference’s name, you can look up this information in the NamedNodeMap of Entity objects returned by the getEntities() method of the DocumentType class. I’ll show you an example of this when we get to the Entity interface. In the meantime, let’s do an example that creates new entity references in the tree.

One common complaint about XML is that it doesn’t support the entity references like &nbsp; and &eacute; developers are accustomed to from HTML. Using DOM, it’s straightforward to replace any inconvenient character with an entity reference as Example 11.14 proves. This program recursively descends the element tree looking for any non-breaking space characters (Unicode code point 0xA0). It replaces any it finds with an entity reference with the name nbsp. To do so it has to split the text node around the non-breaking space.

Example 11.14. Inserting entity references into a document

import org.w3c.dom.*;

public class NBSPUtility {

  // Recursively descend the tree replacing all non-breaking
  // spaces with &nbsp;
  public static void addEntityReferences(Node node) {

    int type = node.getNodeType();
    if (type == Node.TEXT_NODE) {
                // the only type with attributes
      Text text = (Text) node;
      String s = text.getNodeValue();
      int nbsp = s.indexOf('\u00A0'); // finds the first A0
      if (nbsp != -1) {
        Text middle = text.splitText(nbsp);
        Text end = middle.splitText(1);
        Node parent = text.getParentNode();
        Document factory = text.getOwnerDocument();
        EntityReference ref = factory.createEntityReference("nbsp");
        parent.replaceChild(ref, middle);
        addEntityReferences(end); // finds any subsequent A0s
        System.out.println("Added");
      }
    } // end if

    else if (node.hasChildNodes()) {
      NodeList children = node.getChildNodes();
      for (int i = 0; i < children.getLength(); i++) {
        Node child = children.item(i);
        addEntityReferences(child);
      } // end for
    } // end if

  }  // end addEntityReferences()

}

It would be easy enough to make it replace all the Latin-1 characters, or all the characters that have standard entity references in HTML, or some such. You’d just need to keep a table of the characters and their corresponding entity references. You could even build such a table from the entities map available from the DTD.

Although this code runs, the documents it produces are not necessarily well-formed. In particular, only entities defined in the DTD should be used. Assuming that’s the case, then the child list of the entity will be automatically filled by the entity’s replacement text. Unfortunately, however, DOM does not offer any means of defining new entities that are not part of the document’s original DTD.

Prev | Up | Next
The CDATASection Interface | Home | The Attr Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 04, 2002


The Attr Interface
Prev Chapter 11. The Document Object Model Core Next

The Attr Interface

Probably 90% of the time, everything you need to do with attributes can be done purely through the various attribute methods of the Element interface. However, occasionally the Attr interface comes in handy. An Attr object represents an attribute, whether explicitly included in the document or defaulted in from the schema.

Example 11.15 summarizes the Attr interface. Besides the methods inherited from the Node super-interface, Attr includes methods to get the full name of the attribute (including a prefix if any), determine whether or not the attribute was specified in the document or merely defaulted in from the DTD or schema, get the element that contains this attribute, and get the value of the attribute. Entity and character references in the value are resolved, but white space is not normalized. There’s also a method that can change the value of this attribute. This throws a DOMException if the attribute is read-only, as it might be if this attribute belongs to an element in a read-only entity reference node.

Example 11.15. The Attr interface

package org.w3c.dom;

public interface Attr extends Node {

  public String  getName();  // Prefixed name
  public String  getValue();
  public Element getOwnerElement();
  public boolean getSpecified();
  public void    setValue(String value) throws DOMException;

}

The key thing to watch out for when working with attributes in DOM is that they are not considered to be children of the elements that contain them. Furthermore, unlike the XPath data model, the elements that contain them are not their parents either. In DOM attributes have no parents, so invoking getParent() on an Attr always returns null.

In DOM terminology an attribute that is actually placed on the start-tag of an element is called specified. An attribute that is defaulted in from the DTD or schema is not specified. You can tell which is which by invoking the getSpecified() method. This is one of the few things you can do with an Attr object that you can’t do with the owning Element object.

One source of interoperability problems in XML is that some parsers validate and some don’t. If validation were only about checking constraints, this wouldn’t be a big problem. Unfortunately validation does more than that. In particular it assigns default attribute values to elements. These attributes may be seen by a validating parser but not by a non-validating parser. Therefore, for an example of this interface, I’m going to write a program that converts unspecified attributes into specified attributes to improve interoperability. An attribute modified or added programmatically in DOM is always specified, even if it has the same value as the default attribute in the DTD. If you delete an attribute for which the DTD provides a default value, then DOM will insert an attribute node with the default value. Thus it’s impossible to get rid of a default attribute. Example 11.16 demonstrates.[4]

Example 11.16. Specifying all attributes

import org.w3c.dom.*;

public class AttributeUtility {

  // Recursively descend the tree replacing all unspecified
  // attributes with specified attributes
  public static void specifyAttributes(Node node) {

    int type = node.getNodeType();
    if (type == Node.ELEMENT_NODE) {
                // the only type with attributes
      Element element = (Element) node;
      NamedNodeMap attributes = element.getAttributes();
      Document factory = node.getOwnerDocument();
      for (int i = 0; i < attributes.getLength(); i++) {
        Attr attribute = (Attr) attributes.item(i);
        if (!attribute.getSpecified()) {
          // We can't change the specified property of an
          // attribute in DOM2. However, attributes are specified
          // by default, so if we delete the old attribute and
          // add a new one with the same name and value, or
          // change the attribute's value (even  to the same
          // thing) the effect is what we're looking for.
          String name = attribute.getName();
          String value = attribute.getValue();
          Attr specifiedAttribute = factory.createAttribute(name);
          specifiedAttribute.setValue(value);
          element.setAttributeNode(specifiedAttribute);
          // This replaces the old attribute with the same name.
        }
      } // end for
    } // end if

    if (node.hasChildNodes()) {
      NodeList children = node.getChildNodes();
      for (int i = 0; i < children.getLength(); i++) {
        Node child = children.item(i);
        specifyAttributes(child);
      } // end for
    } // end if

  }  // end specifyAttributes()

}

This program simply recursively descends the tree beneath a node, looking for elements. It scans each element’s attributes. If any of those attributes are not specified, then the attribute is deleted and replaced with a copy.

DOM2 is missing one crucial feature for working with attributes. It cannot tell you the type of the attribute as declared in the DTD. That is, you cannot find out if an attribute is CDATA, ID, IDREF, NMTOKEN, and so forth.

[4] I admit this example is a little contrived. Reading each value and writing the same value back out again would be easier.

Prev | Up | Next
The EntityReference Interface | Home | The ProcessingInstruction Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 12, 2002


The ProcessingInstruction Interface
Prev Chapter 11. The Document Object Model Core Next

The ProcessingInstruction Interface

The ProcessingInstruction interface represents a processing instruction such as <?xml-stylesheet type="text/css" href="order.css"?> or <?php echo "Hello World";?>.

Example 11.17 summarizes the ProcessingInstruction interface. This interface adds methods to get the target and the data of the processing instruction as strings. Even if the data has a pseudo-attribute format as in <?xml-stylesheet type="text/css" href="order.css"?>. DOM doesn’t recognize that. For this processing instruction the target is xml-stylesheet and the data is type="text/css" href="order.css".

Example 11.17. The ProcessingInstruction interface

package org.w3c.dom;

public interface ProcessingInstruction extends Node {

  public String getTarget();
  public String getData();
  public void   setData(String data) throws DOMException;

}

As usual, ProcessingInstruction objects also have all the methods of the Node super-interface such as getNodeName() and getNodeValue(). The value of a processing instruction is its data. However, processing instructions do not have children, so Node methods like getFirstChild() return null, and methods like appendChild() throw a DOMException with the code HIERARCHY_REQUEST_ERR.

As an example, let’s extend the earlier XLinkSpider program so that it respects robots processing instructions. Such an instruction looks like this, and appears in the prolog of an XML document:

<?robots index="yes" follow="no"?>

The semantics of this instruction is deliberately similar to the robots META tag in HTML. That is, follow="yes" means robots should follow links they find in this page, follow="no" means they shouldn’t. Similarly, index="yes" means search engines should include this page, and index="no" means they shouldn’t.

Like many processing instructions, the syntax is based on pseudo-attributes. DOM doesn’t provide any means to parse these, even though it’s a very common format for processing instructions. However you can fake DOM out. What I’m going to do is extract the target and data of the processing instruction and use them to form a string that has this format:

<target data/>

In other words, a processing instruction like <?robots index="yes" follow="no"?> is going to turn into a String like <robots index="yes" follow="no" />. This string is in turn a well-formed XML document that can be parsed and its attributes extracted. Admittedly, this approach is very circuitous and probably not optimally efficient. However, it’s a lot easier to code and explain than writing your own mini-parser just to handle pseudo-attributes. Example 11.18 is a simple utility class that implements this hack. The parsing is completely hidden inside the constructor, so if this is too offensive to your sensibilities, you can replace it with more appropriate code without changing the public interface. Since this class is quite useful in practice, not merely an example for this book, I’ve placed it in the com.macfaq.xml package. Don’t forget to configure your class and source paths appropriately when compiling it.

Example 11.18. Reading PseudoAttributes from a ProcessingInstruction

package com.macfaq.xml;

import org.w3c.dom.*;
import javax.xml.parsers.*;
import org.xml.sax.*;
import java.io.*;

public class PseudoAttributes {

  private NamedNodeMap pseudo;

  public PseudoAttributes(ProcessingInstruction pi)
   throws SAXException {

    StringBuffer sb = new StringBuffer("<");
    sb.append(pi.getTarget());
    sb.append(" ");
    sb.append(pi.getData());
    sb.append("/>");
    StringReader reader = new StringReader(sb.toString());
    InputSource source = new InputSource(reader);
    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      DocumentBuilder parser = factory.newDocumentBuilder();

      // This line will throw a SAXException if the processing
      // instruction does not use pseudo-attributes
      Document doc = parser.parse(source);
      Element root = doc.getDocumentElement();
      pseudo = root.getAttributes();

    }
    catch (FactoryConfigurationError e) {
      // I don't absolutely need to catch this, but I hate to
      // throw an Error for no good reason.
      throw new SAXException(e.getMessage());
    }
    catch (SAXException e) {
      throw e;
    }
    catch (Exception e) {
      throw new SAXException(e);
    }

  }

  // delegator methods
  public Attr item(int index) {
    return (Attr) pseudo.item(index);
  }

  public int getLength() {
    return pseudo.getLength();
  }

  public String getValue(String name) {
    Attr att = (Attr) pseudo.getNamedItem(name);
    if (att == null) return "";
    return att.getValue();
  }

}

This class makes it easy for the earlier DOMSpider program to recognize the robots processing instruction. I won’t repeat the entire program, most of which hasn’t changed. The relevant change is in the spider() method. It now has to look for a robots processing instruction in each document and use that to decide whether or not to call process() (index="yes|no") and/or findLinks() (follow="yes|no").

  public void spider(String systemID) {
    currentDepth++;
    try {
      if (currentDepth < maxDepth) {
        Document document = parser.parse(systemID);

        // Look for a robots PI with follow="no"
        boolean index = true;
        boolean follow = true;
        NodeList children = document.getChildNodes();
        for (int i = 0; i < children.getLength(); i++) {
          Node child = children.item(i);
          int type = child.getNodeType();
          if (type == Node.PROCESSING_INSTRUCTION_NODE) {
            ProcessingInstruction pi
             = (ProcessingInstruction) child;
            if (pi.getTarget().equals("robots")) {
               PseudoAttributes pseudo = new PseudoAttributes(pi);
               if (pseudo.getValue("index").equals("no")) {
                 index = false;
               }
               if (pseudo.getValue("follow").equals("no")) {
                 follow = false;
               }
            }
          }
        } // end for

        if (index) process(document, systemID);

        if (follow) {
          Vector toBeVisited = new Vector();
          // search the document for uris,
          // store them in vector, and print them
          findLinks(
           document.getDocumentElement(), toBeVisited, systemID);

          Enumeration e = toBeVisited.elements();
          while (e.hasMoreElements()) {
            String uri = (String) e.nextElement();
            visited.add(uri);
            spider(uri);
          } // end while
        } // end if

      }

    }
    catch (SAXException e) {
      // Couldn't load the document,
      // probably not well-formed XML, skip it
    }
    catch (IOException e) {
      // Couldn't load the document,
      // likely network failure, skip it
    }
    finally {
      currentDepth--;
      System.out.flush();
    }

  }
Prev | Up | Next
The Attr Interface | Home | The Comment Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 08, 2002


The Comment Interface
Prev Chapter 11. The Document Object Model Core Next

The Comment Interface

XML comments don’t have a lot of structure. They’re really just some undifferentiated text inside <!-- and -->. Therefore, the Comment interface, shown in Example 11.19, is a subinterface of CharacterData and shares all its method with that interface. However, your code can use the type to determine that a node is a comment, and treat it appropriately. Serializers will be smart enough to output a Comment with the right markup around it.

Example 11.19. The Comment interface

package org.w3c.dom;

public interface Comment extends CharacterData {

}

Earlier in Chapter 7, I demonstrated a SAX program that read comments. Now in Example 11.20 you can see the DOM equivalent. The approach is different— actively walking a tree instead of passively receiving events—but the effect is the same, printing the contents of comments and only comments on System.out.

Example 11.20. Printing comments

import javax.xml.parsers.*;
import org.w3c.dom.*;
import org.xml.sax.SAXException;
import java.io.IOException;

public class DOMCommentReader {

  // note use of recursion
  public static void printComments(Node node) {

    int type = node.getNodeType();
    if (type == Node.COMMENT_NODE) {
      Comment comment = (Comment) node;
      System.out.println(comment.getData());
      System.out.println();
    }
    else {
      if (node.hasChildNodes()) {
        NodeList children = node.getChildNodes();
        for (int i = 0; i < children.getLength(); i++) {
          printComments(children.item(i));
        }
      }
    }

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java DOMCommentReader URL");
      return;
    }

    String url = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      DocumentBuilder parser = factory.newDocumentBuilder();

      // Read the document
      Document document = parser.parse(url);

      // Process the document
      DOMCommentReader.printComments(document);

    }
    catch (SAXException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check " + url
      );
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  } // end main

}

Here’s the result of running this program on the XML Schema Datatypes specification:

D:\books\XMLJAVA>java DOMCommentReader
 http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/datatypes.xml
  commenting these out means only that they won't show up in the
    stylesheet generated "Revisions from previous draft" appendix

 Changes before Sept public draft commented out...
<sitem>
19990521: PVB: corrected definition of length and maxLengths
facet for strings to be in terms of <emph>characters</emph>
not <emph>bytes</emph>
</sitem>
<sitem>
19990521: PVB: removed issue "other-date-representations".
We don't want other separators, left mention of aggregate reps
for dates as an ednote.
</sitem>
<sitem>
19990521: PVB: fixed "holidays" example, "-0101" ==> "==0101"
(where == in the correction should be two hyphens, but that would
not allow us to comment out this sitem)
…

It’s not obvious from this output sample, but there is a big difference between the behavior of the SAX and DOM versions of this program. The SAX version begins producing output almost immediately because it works in streaming mode. However, the DOM version first has to read the entire document from the remote URL, parse it, and only then begin walking the tree to look for comments. The SAX and DOM versions are both limited by the speed of the network connection so they both take about the same amount of time to run on the same input data. However, the SAX version begins returning results much more quickly than the DOM version which doesn’t present any results until the entire document has been read. This may not be a big concern in a batch-mode application, but it can be very important when there is a human user. The SAX version will feel a lot more responsive.

Prev | Up | Next
The ProcessingInstruction Interface | Home | The DocumentType Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


The DocumentType Interface
Prev Chapter 11. The Document Object Model Core Next

The DocumentType Interface

The DocumentType interface represents a document’s document type declaration. That is, it’s the in-memory representation of this construct:

<!DOCTYPE mml:math PUBLIC "-//W3C//DTD MathML 2.0//EN"
 "http://www.w3.org/TR/MathML2/dtd/mathml2.dtd" [
  <!ENTITY % MATHML.prefixed "INCLUDE">
  <!ENTITY % MATHML.prefix "mml">
]>

Each document type declaration has four parts, the last three of which are optional:

  • The root element name (mml:math in the above example)

  • The public ID (-//W3C//DTD MathML 2.0//EN in the above example)

  • The system ID (http://www.w3.org/TR/MathML2/dtd/mathml2.dtd in the above example)

  • The internal DTD subset (Everything between the [ and the ])

It’s important to note that the document type declaration is not the same thing as the document type definition. The document type declaration points to the document type definition, and may sometime contain the document type defintion or part of it as the internal DTD subset. However, the document type declaration and the document type definition are two different things. DOM2 only provides access to the document type declaration. It does not tell you what the document type definition says. The acronym DTD is used only for the document type definition.

Example 11.21 summarizes DOM’s DocumentType interface. It has methods to get the root element name, the public ID, the system ID, the internal DTD subset, and maps of the entities and notations declared in the DTD this document type declaration points to.

Example 11.21. The DocumentType interface

package org.w3c.dom;

public interface DocumentType extends Node {

  public String       getName();
  public NamedNodeMap getEntities();
  public NamedNodeMap getNotations();
  public String       getPublicId();
  public String       getSystemId();
  public String       getInternalSubset();

}

All of these properties are read-only. That is, there are getter methods but no corresponding setter methods. You cannot change the name, public ID, system ID or anything else about a DocumentType object. Once the parser has read it, it’s final.

A DocumentType object is created by a DOMImplementation object and assigned to a document when the Document object is created. After that point, a document’s DocumentType cannot be changed.

The next two sections will provide some examples of this interface.

Prev | Up | Next
The Comment Interface | Home | The Entity Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 29, 2002


The Entity Interface
Prev Chapter 11. The Document Object Model Core Next

The Entity Interface

The Entity interface represents a parsed or unparsed general entity declared in a document’s DTD. (DOM does not expose parameter entities.) A map of the entities declared in a document is available from the getEntities() method of the DocumentType interface. However, entities are not part of the tree structure. The parent of an entity is always null.

An Entity object represents the actual storage unit. It does not represent the entity reference such as &Omega; or &copyright; that appears in the instance document, but rather the replacement text that reference points to. For parsed entities that the XML parser has resolved, the descendants of the Entity object form a read-only tree containing the XML markup the entity reference stands for. For unparsed entities and external entities that the XML parser has not read, the Entity object has no children.

Example 11.22 summarizes the Entity interface. This interface includes methods to get the public ID, system ID, and notation name for the entity. These methods all return null if that property is not applicable to this entity. To get the replacement text of an entity use the methods Entity inherits from its Node super-interface such as hasChildNodes() and getFirstChild().

Example 11.22. The Entity interface

package org.w3c.dom;

public interface Entity extends Node {

  public String getPublicId();
  public String getSystemId();
  public String getNotationName();

}

For an example, let’s look at a program that walks the document looking for entity references. Every time it sees one it prints out that reference’s name, public ID, and system ID. To do this, it has to look up the entity reference’s name in the entities map returned by the getEntities() of the DocumentType interface. A java.util.Set keeps track of which entities have been printed so it won’t print any entity more than once.

Example 11.23. Listing parsed entities used in the document

import javax.xml.parsers.*;
import org.w3c.dom.*;
import org.xml.sax.SAXException;
import java.io.IOException;
import java.util.*;

public class EntityLister {

  // Store the entities that have already been printed
  private Set          printed = new HashSet();
  private NamedNodeMap entities;

  // Recursively descend the tree
  public void printEntities(Document doc) {

    DocumentType doctype = doc.getDoctype();
    entities = doctype.getEntities();
    seekEntities(doc);

  }

  // note use of recursion
  private void seekEntities(Node node) {

    int type = node.getNodeType();
    if (type == Node.ENTITY_REFERENCE_NODE) {
      EntityReference ref = (EntityReference) node;
      printEntityReference(ref);
    }

    if (node.hasChildNodes()) {
      NodeList children = node.getChildNodes();
      for (int i = 0; i < children.getLength(); i++) {
        seekEntities(children.item(i));
      }
    }

  }

  private void printEntityReference(EntityReference ref) {

    String name = ref.getNodeName();
    if (!printed.contains(name)) {

      Entity entity   = (Entity) entities.getNamedItem(name);
      String publicID = entity.getPublicId();
      String systemID = entity.getSystemId();

      System.out.print(name + ": ");
      if (publicID != null) System.out.print(publicID + " ");
      if (systemID != null) System.out.print(systemID + " ");
      else { // Internal entities do not have system IDs
        System.out.print("internal entity");
      }
      System.out.println();

      printed.add(name);
    }

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java EntityLister URL");
      return;
    }
    String url = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();

      // By default JAXP does not include entity reference nodes
      // in the tree. You have to explicitly request them by
      // telling DocumentBuilderFactory not to expand entity
      // references.
      factory.setExpandEntityReferences(false);
      DocumentBuilder parser = factory.newDocumentBuilder();

      // Read the document
      Document document = parser.parse(url);

      // Print the entities
      EntityLister lister = new EntityLister();
      lister.printEntities(document);

    }
    catch (SAXException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not read " + url
      );
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  } // end main

}

Mostly this is fairly straightforward tree-walking code of the sort you’ve seen several times before. However, you should note that by default JAXP DocumentBuilder objects do not put any entity reference nodes in the trees they build. To get these, the expand entity references property must be explicitly set to false on the DocumentBuilderFactory that creates the DocumentBuilder.

Here’s the output when I ran this across the DocBook source for this chapter. All the entity references used here are internally defined references to single hard-to-type characters like curly quotes and the em dash.

D:\books\XMLJAVA>java EntityLister file://D/books/XMLJava/ch11.xml
rsquo: internal entity
mdash: internal entity
hellip: internal entity
Prev | Up | Next
The DocumentType Interface | Home | The Notation Interface
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 10, 2002


The Notation Interface
Prev Chapter 11. The Document Object Model Core Next

The Notation Interface

The Notation interface represents a notation declared in the DTD. It is not part of the tree. A Notation object has no parent and no children. The only way to access a document’s notations is through the getNotations() method of the DocumentType object.

Note

Notations are very uncommon in practice. You may want to skip this section unless you discover a specific need for it.

Example 11.24 summarizes the Notation interface. This interface has methods to get the system and public ID for the notation. Either of these IDs may be null. As usual, Notation objects also have all the methods of the Node super-interface such as getNodeName() and getNodeValue().

Example 11.24. The Notation interface

package org.w3c.dom;

public interface Notation extends Node {

  public String getPublicId();
  public String getSystemId();

}

Example 11.25 is a simple program that lists all the notations declared in a document’s DTD. To determine whether these were actually used anywhere in the document, you’d need to compare them to all the processing instruction targets, notation type attributes, and unparsed entities to see if they matched up anywhere.

Example 11.25. Listing the Notations declared in a DTD

import javax.xml.parsers.*;
import org.w3c.dom.*;
import org.xml.sax.SAXException;
import java.io.IOException;

public class NotationLister {

  // No recursion for a change. We don't need to walk the tree.
  public static void listNotations(Document doc) {

    DocumentType doctype   = doc.getDoctype();
    NamedNodeMap notations = doctype.getNotations();
    for (int i = 0; i < notations.getLength(); i++) {

      Notation notation = (Notation) notations.item(i);

      String name       = notation.getNodeName();
      String publicID   = notation.getPublicId();
      String systemID   = notation.getSystemId();

      System.out.print(name + ": ");
      if (publicID != null) System.out.print(publicID + " ");
      if (systemID != null) System.out.print(systemID + " ");
      System.out.println();

    }

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java NotationLister URL");
      return;
    }
    String url = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      DocumentBuilder parser = factory.newDocumentBuilder();

      // Read the document
      Document document = parser.parse(url);

      // Process the document
      listNotations(document);

    }
    catch (SAXException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check " + url
      );
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  } // end main

}

DocBook is the only XML application I’m aware of that even declares any notations (and that’s really only because of legacy compatibility issues with DocBook’s earlier SGML incarnation). This book is written in DocBook, so I ran NotationLister across one of its chapters. Here’s the output:

D:\books\XMLJAVA>java NotationLister masterbook.xml
BMP: +//ISBN 0-7923-9432-1::Graphic Notation//NOTATION Microsoft
 Windows bitmap//EN
CGM-BINARY: ISO 8632/3//NOTATION Binary encoding//EN
CGM-CHAR: ISO 8632/2//NOTATION Character encoding//EN
CGM-CLEAR: ISO 8632/4//NOTATION Clear text encoding//EN
DITROFF: DITROFF
DVI: DVI
EPS: +//ISBN 0-201-18127-4::Adobe//NOTATION PostScript Language
 Ref. Manual//EN
EQN: EQN
FAX: -//USA-DOD//NOTATION CCITT Group 4 Facsimile Type 1
 Untiled Raster//EN
GIF: GIF
GIF87a: -//CompuServe//NOTATION Graphics Interchange Format
 87a//EN
GIF89a: -//CompuServe//NOTATION Graphics Interchange Format
 89a//EN
IGES: -//USA-DOD//NOTATION (ASME/ANSI Y14.26M-1987) Initial
 Graphics Exchange Specification//EN
JPEG: JPG
JPG: JPG
PCX: +//ISBN 0-7923-9432-1::Graphic Notation//NOTATION ZSoft PCX
 bitmap//EN
PIC: PIC
PNG: http://www.w3.org/TR/REC-png
PS: PS
SGML: ISO 8879:1986//NOTATION Standard Generalized Markup
 Language//EN
TBL: TBL
TEX: +//ISBN 0-201-13448-9::Knuth//NOTATION The TeXbook//EN
TIFF: TIFF
WMF: +//ISBN 0-7923-9432-1::Graphic Notation//NOTATION Microsoft
Windows Metafile//EN
WPG: WPG
linespecific: linespecific

I had to add a few line breaks to fit the output on the page, but otherwise the result would be the same given any DocBook document because the notations come from the DTD, not the instance document.

Prev | Up | Next
The Entity Interface | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 04, 2002


Summary
Prev Chapter 11. The Document Object Model Core Next

Summary

The org.w3c.dom package includes 13 different subclasses of Node that represent the different possible components of a DOM tree:

Document

The root of the DOM tree; acts as a factory class for the other kinds of nodes; offers access to the root element, document type declaration, and DOM implementation.

DocumentType

Represents the document type declaration; provides access to the notations and entities declared in the document’s DTD.

Element

Represents an element including its local name, prefixed name, namespace URI, and attributes

CharacterData

A common super-interface for Text, CDATASection, and Comment; provides methods to read and modify the text content of these nodes.

Text

Represents a text node in the DOM tree; has a method to split a text node into two text nodes

Attr

Represents an attribute of an element; not part of the tree; has methods to get the name, value, owner element, and default status of an attribute and to change the value of an attribute (but not the name). Most of the time the attribute-related methods in the Element interface are used rather than working directly with attribute nodes.

Comment

Represents a comment node in the DOM tree; inherits all its methods from the super-interfaces CharacterData and Node

ProcessingInstruction

Represents a processing instruction node in the DOM tree; has methods to get the target and to get and set the data

CDATASection

Represents a CDATA section node in the DOM tree; inherits all its methods from the super-interfaces Text, CharacterData and Node

EntityReference

Represents an entity reference node in the DOM tree. The children of this node are formed from the replacement text of the entity. Some parsers resolve entity references without putting any entity reference nodes in the tree.

Entity

Represents a parsed or unparsed general entity declared in the DTD. Includes methods to get the public and system ID of the entity. The children of an Entity object are formed from the replacement text of the entity. Entity objects are only accessible through the NamedNodeMap returned by the getEntities() method in the DocumentType interface.

Notation

Represents a notation declared in the DTD; includes methods to get the public and/or system ID of the notation; only accessible through the NamedNodeMap returned by the getNotations() method in the DocumentType interface.

DocumentFragment

A place holder for a part of a document that contains multiple sibling nodes that can be moved as a group. It differs from other nodes in that when a document fragment is inserted into a node, the children of the document fragment are placed in that node, but not the document fragment itself.

Each of these interfaces inherits the methods of the Node interface as well as methods providing unique functionality for that particular kind of node.

Prev | Up | Next
The Notation Interface | Home | Chapter 12. The DOM Traversal Module
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 26, 2002


Chapter 12. The DOM Traversal Module
Prev Next

Chapter 12. The DOM Traversal Module

Table of Contents

NodeIterator
Constructing NodeIterators with DocumentTraversal
Liveness
Filtering by Node Type
NodeFilter
TreeWalker
Summary

The examples in the last few chapters have duplicated quite a bit of tree walking code. Some of them searched for particular information. Others modified documents in memory. What they all had in common was that they navigated a tree from the root to the deepest leaf element in document order. This is an extremely common pattern in DOM programs.

The org.w3c.dom.traversal package is a collection of utility interfaces that implement most of the logic needed to traverse a DOM tree. These include NodeIterator, NodeFilter, TreeWalker and DocumentTraversal. DOM implementations are not required to support these interfaces, but many do including Oracle and Xerces. (Crimson does not. GNU JAXP supports NodeIterator but not TreeWalker.) By reusing these classes you can simplify your programs a great deal and save yourself a significant amount of work.

NodeIterator

The NodeIterator utility interface extracts a subset of the nodes in a DOM document and presents them as a list arranged in document order. In other words, the nodes appear in the order you’d find them in a depth-first, pre-order traversal of the tree. That is,

  • The document node comes first.

  • Parents come before their children. Ancestors come before their descendants.

  • Sibling nodes appear in the same order as their start-tags appear in the text representation of the document.

In other words, this is pretty much the order you’d expect by just reading an XML document from beginning to end. As soon as you see the first character of text from a node, that node is counted.

You can iterate through this list without concerning yourself with the tree structure of the XML document. For many operations this flatter view is more convenient than the hierarchical tree view. For example, a spell checker can check all text nodes one at a time. An outline program can extract the headings in an XHTML document while ignoring everything else. You can do all this by iterating though a list without having to write recursive methods.

Example 12.1 summarizes the NodeIterator interface. The first four getter methods simply tell you how the iterator is choosing from all the available nodes in the document. The nextNode() and previousNode() methods move forwards and backwards in the list and return the requested node. Finally, the detach() method cleans up after the iterator when you’re done with it. It’s analogous to closing a stream.

Example 12.1. The NodeIterator interface

package org.w3c.dom.traversal;

public interface NodeIterator {

    public Node       getRoot();
    public int        getWhatToShow();
    public NodeFilter getFilter();
    public boolean    getExpandEntityReferences();

    public Node       nextNode() throws DOMException;
    public Node       previousNode() throws DOMException;

    public void       detach();

}

As you see, the NodeIterator interface provides only the most basic methods for an iterator. Each iterator can be thought of as having a cursor which is initially positioned before the first node in the list. The nextNode() method returns the node immediately following the cursor and advances the cursor one space. The previousNode() method returns the node immediately before the cursor and backs the cursor up one space. If the iterator is positioned at the end of the list, nextNode() returns null. If the iterator is positioned at the beginning of the list, previousNode() returns null. For example, given a NodeIterator variable named iterator positioned at the beginning of its list, this code fragment prints the names of all the nodes:

Node node;
while ((node = iterator.nextNode()) != null) {
  System.out.println(node.getNodeName());
}

Note

Design pattern aficionados will have recognized this as instance of the iterator pattern (as if the name didn’t already give it away). More precisely, it’s a robust, external iterator. Robust means that the iterator still works even if its backing data structure (the Document object) changes underneath it. External means that client code is responsible for moving the iterator from one node to the next, rather than having the iterator move itself.

Constructing NodeIterators with DocumentTraversal

Not all DOM implementations are guaranteed to support the traversal module, though most do. You can check this with hasFeature("traversal", "2.0") in the DOMImplementation class. For example,

if (!impl.hasFeature("traversal", "2.0")) {
  System.err.println(
   "A DOM implementation that supports traversal is required.");
  return;
}

Assuming the implementation does support traversal, the Document implementation class also implements the DocumentTraversal interface. This factory interface, shown in Example 12.2, allows you to create new NodeIterator and TreeWalker objects which traverse the nodes in that document.

Example 12.2. The DocumentTraversal factory interface

package org.w3c.dom.traversal;

public interface DocumentTraversal {

  public NodeIterator createNodeIterator(Node root,
   int whatToShow, NodeFilter filter,
   boolean entityReferenceExpansion) throws DOMException;

  public TreeWalker createTreeWalker(Node root,
   int whatToShow, NodeFilter filter,
   boolean entityReferenceExpansion) throws DOMException;

}

Thus, to create a NodeIterator you cast the Document object you want to iterate over to DocumentTraversal, and then invoke its createNodeIterator() method. This method takes four arguments:

root

The Node in the document which the iterator starts from. Only this node and its descendants are traversed by the iterator. This means you can easily design iterators that iterate over a subtree of the entire document. For instance, by passing in the root element, you could skip everything in the document’s prolog and epilog.

whatToShow

An int bitfield constant specifying the node types the iterator will include. These constants are:

  • NodeFilter.SHOW_ELEMENT = 1

  • NodeFilter.SHOW_ATTRIBUTE = 2

  • NodeFilter.SHOW_TEXT = 4

  • NodeFilter.SHOW_CDATA_SECTION = 8

  • NodeFilter.SHOW_ENTITY_REFERENCE = 16

  • NodeFilter.SHOW_ENTITY = 32

  • NodeFilter.SHOW_PROCESSING_INSTRUCTION = 64

  • NodeFilter.SHOW_DOCUMENT = 128

  • NodeFilter.SHOW_DOCUMENT_TYPE = 256

  • NodeFilter.SHOW_DOCUMENT_FRAGMENT = 512

  • NodeFilter.SHOW_NOTATION = 1024

  • NodeFilter.SHOW_ALL = 0xFFFFFFFF

These are all powers of two so that they can be combined with the bitwise operators. For example, if you wanted to iterate over text nodes and CDATA section nodes, you would pass NodeFilter.SHOW_TEXT | NodeFilter.SHOW_CDATA_SECTION as whatToShow.

filter

A NodeFilter against which all nodes in the subtree will be compared. Only nodes that pass the filter will be let through. By implementing this interface, you can define more specific filters such as “all elements that have xlink:type="simple" attributes” or “all text nodes that contain the word fnord”. You can pass null to indicate no custom filtering.

entityReferenceExpansion

Pass true if you want the iterator to descend through the children of entity reference nodes, false otherwise. Generally, this should be set to true.

For example, the last chapter demonstrated a comment reader program that recursively descended an XML tree, printing out all the comment nodes that were found. A NodeIterator makes it possible to write the program non-recursively. When creating the iterator, the root argument is the document node, whatToShow is NodeFilter.SHOW_COMMENT, the node filter is null, and entityReferenceExpansion is true. Example 12.3 demonstrates.

Example 12.3. Using a NodeIterator to extract all the comments from a document

import javax.xml.parsers.*;
import org.w3c.dom.*;
import org.w3c.dom.traversal.*;
import org.xml.sax.SAXException;
import java.io.IOException;

public class CommentIterator {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java DOMCommentReader URL");
      return;
    }

    String url = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      DocumentBuilder parser = factory.newDocumentBuilder();

      // Check for the traversal module
      DOMImplementation impl = parser.getDOMImplementation();
      if (!impl.hasFeature("traversal", "2.0")) {
        System.out.println(
         "A DOM implementation that supports traversal is required."
        );
        return;
      }

      // Read the document
      Document doc = parser.parse(url);

      // Create the NodeIterator
      DocumentTraversal traversable = (DocumentTraversal) doc;
      NodeIterator iterator = traversable.createNodeIterator(
       doc, NodeFilter.SHOW_COMMENT, null, true);

      // Iterate over the comments
      Node node;
      while ((node = iterator.nextNode()) != null) {
        System.out.println(node.getNodeValue());
      }

    }
    catch (SAXException e) {
      System.out.println(e);
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check " + url
      );
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  } // end main

}

You can decide for yourself whether or not you prefer the explicit recursion and tree-walking of Example 11.20 in the last chapter or the hidden recursion of CommentIterator here. With a decent implementation, there really shouldn’t be any noticeable performance penalty, so feel free to use whichever feels more natural to you.

Liveness

Node iterators are live. That is, if the document changes while the program is walking the tree, the iterator retains its state. For instance, let’s suppose the program is at node C of a node iterator that’s walking through nodes A, B, C, D, and E in that order. If you delete node D, and then call nextNode(), you’ll get node E. If you add node Z in between B and C and then call previousNode(), you’ll get node Z. The iterator’s current position is always between two nodes (or before the first node or after the last node) but never on a node. Thus it is not invalidated by deleting the current node.

For example, this method deletes all the comments in its Document argument. When the method returns all the comments have been removed.

  public static void deleteComments(Document doc) {

    // Create the NodeIterator
    DocumentTraversal traversable = (DocumentTraversal) doc;
    NodeIterator iterator = traversable.createNodeIterator(
     doc, NodeFilter.SHOW_COMMENT, null, true);

    // Iterate over the comments
    Node comment;
    while ((comment = iterator.nextNode()) != null) {
      Node parent = comment.getParentNode();
      parent.removeChild(comment);
    }

  }

This method changes the original Document object. It does not change the XML file from which the Document object was created unless you specifically write the changed document back out into the original file after the comments have been deleted.

Filtering by Node Type

You can combine the various flags for whatToShow with the bitwise or operator. For example, the previous chapter used a rather convoluted recursive getText() method in the ExampleExtractor program to accumulate all the text from both text and CDATA section nodes within an element. Example 12.4 shows how NodeIterator can be used to accomplish this task in a much more straightforward fashion.

Example 12.4. Using a NodeIterator to retrieve the complete text content of an element

import org.w3c.dom.*;
import org.w3c.dom.traversal.*;

public class TextExtractor {

  public static String getText(Node node) {

    if (node == null) return "";

    // Set up the iterator
    Document doc = node.getOwnerDocument();
    DocumentTraversal traversable = (DocumentTraversal) doc;
    int whatToShow
     = NodeFilter.SHOW_TEXT | NodeFilter.SHOW_CDATA_SECTION;
    NodeIterator iterator = traversable.createNodeIterator(node,
     whatToShow, null, true);

    // Extract the text
    StringBuffer result = new StringBuffer();
    Node current;
    while ((current = iterator.nextNode()) != null) {
      result.append(current.getNodeValue());
    }
    return result.toString();

  }

}

I’ll be reusing this class a little later on. Something like this should definitely be in your toolbox for whenever you need to extract the text content of an element.

Note

DOM Level 3 is going to add an almost equivalent getTextContent() method to the Node interface:

public String getTextContent()
    throws DOMException;

The only difference is that this method will not operate on Document objects whereas TextExtractor.getText() will.

Prev | Up | Next
Summary | Home | NodeFilter
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 19, 2002


NodeFilter
Prev Chapter 12. The DOM Traversal Module Next

NodeFilter

The whatToShow argument allows you to iterate over only certain node types in a subtree. However, suppose you want to go beyond that. For example, you may have a program that reads XHTML documents and extracts all heading elements but ignores everything else. Or perhaps, you want to find all SVG content in a document, or all the GIFT elements whose price attribute has a value greater than $10.00. Or perhaps you want to find those SKU elements containing an ID of a product that needs to be reordered, as determined by consulting an external database. All of these tasks and many more besides can be implemented through node filters on top of a NodeIterator or a TreeWalker.

Example 12.5 summarizes the NodeFilter interface. You implement this interface in a class of your own devising. The acceptNode() method contains the the custom logic that decides whether any given node passes the filter or not. This method can return one of the three named constants NodeFilter.FILTER_ACCEPT, NodeFilter.FILTER_REJECT, or NodeFilter.FILTER_SKIP to indicate what it wants to do with that node.

Example 12.5. The NodeFilter interface

package org.w3c.dom.traversal;

public interface NodeFilter {

  // Constants returned by acceptNode
  public static final short FILTER_ACCEPT = 1;
  public static final short FILTER_REJECT = 2;
  public static final short FILTER_SKIP   = 3;

  // Constants for whatToShow
  public static final int SHOW_ALL               = 0xFFFFFFFF;
  public static final int SHOW_ELEMENT           = 0x00000001;
  public static final int SHOW_ATTRIBUTE         = 0x00000002;
  public static final int SHOW_TEXT              = 0x00000004;
  public static final int SHOW_CDATA_SECTION     = 0x00000008;
  public static final int SHOW_ENTITY_REFERENCE  = 0x00000010;
  public static final int SHOW_ENTITY            = 0x00000020;
  public static final int SHOW_PROCESSING_INSTRUCTION
   = 0x00000040;
  public static final int SHOW_COMMENT           = 0x00000080;
  public static final int SHOW_DOCUMENT          = 0x00000100;
  public static final int SHOW_DOCUMENT_TYPE     = 0x00000200;
  public static final int SHOW_DOCUMENT_FRAGMENT = 0x00000400;
  public static final int SHOW_NOTATION          = 0x00000800;

  public short acceptNode(Node n);

}

For iterators, there are really only two options for the return value of acceptNode(), FILTER_ACCEPT and FILTER_SKIP. NodeIterator treats FILTER_REJECT the same as FILTER_SKIP. (Tree walkers do make a distinction between these two.) Rejecting a node prevents it from appearing in the list, but does not prevent its children and descendants from appearing. They will be tested separately.

The NodeFilter does not override whatToShow. They work in concert. For example, whatToShow can limit the iterator to only elements. Then the acceptNode() method can confidently cast every node that’s passed to it to Element without first checking its node type.

To configure an iterator with a filter, pass the NodeFilter object to the createNodeIterator() method. The NodeIterator then passes each potential candidate node to the acceptNode() method to decide whether or not to include it in the iterator.

For an example, let’s revisit last chapter’s DOMSpider program. That program needed to recurse through the entire document, looking at each and every node to see whether or not it was an element and, if it was, whether or not it had an xlink:type attribute with the value simple. We can write that program much more simply using a NodeFilter to find the simple XLinks and a NodeIterator to walk through them. Example 12.6 demonstrates the necessary filter.

Example 12.6. An implementation of the NodeFilter interface

import org.w3c.dom.traversal.NodeFilter;
import org.w3c.dom.*;

public class XLinkFilter implements NodeFilter {

  public static String XLINK_NAMESPACE
   = "http://www.w3.org/1999/xlink";

  public short acceptNode(Node node) {

    Element candidate = (Element) node;
    String type
     = candidate.getAttributeNS(XLINK_NAMESPACE, "type");
    if (type.equals("simple")) return FILTER_ACCEPT;
    return FILTER_SKIP;

  }

}

Here’s a spider() method that has been revised to take advantage of NodeIterator and this filter. This can replace both the spider() and findLinks() methods of the previous version. The filter replaces the isSimpleLink() method. The code is quite a bit simpler than the version in the last chapter.

  public void spider(String systemID) {
    currentDepth++;
    try {
      if (currentDepth < maxDepth) {
        Document document = parser.parse(systemID);
        process(document, systemID);

        Vector uris = new Vector();
        // search the document for uris,
        // store them in vector, and print them
        DocumentTraversal traversal
         = (DocumentTraversal) document;
        NodeIterator xlinks = traversal.createNodeIterator(
          document.getDocumentElement(),// start at root element
          NodeFilter.SHOW_ELEMENT,      // only see elements
          new XLinkFilter(),            // only see simple XLinks
          true                          // expand entities
        );

        Element xlink;
        while ((xlink = (Element) xlinks.nextNode()) != null) {
          String uri = xlink.getAttributeNS(XLINK_NAMESPACE,
           "href");
          if (!uri.equals("")) {
            try {
              String wholePage = absolutize(systemID, uri);
              if (!visited.contains(wholePage)
               && !uris.contains(wholePage)) {
                uris.add(wholePage);
              }
            }
            catch (MalformedURLException e) {
              // If it's not a good URL, then we can't spider it
              // anyway, so just drop it on the floor.
            }
          } // end if
        } // end while
        xlinks.detach();

        Enumeration e = uris.elements();
        while (e.hasMoreElements()) {
          String uri = (String) e.nextElement();
          visited.add(uri);
          spider(uri);
        }

      }

    }
    catch (SAXException e) {
      // Couldn't load the document,
      // probably not well-formed XML, skip it
    }
    catch (IOException e) {
      // Couldn't load the document,
      // likely network failure, skip it
    }
    finally {
      currentDepth--;
      System.out.flush();
    }

  }

There is, however, one feature the earlier version had that this NodeIterator based variant doesn’t have. Last chapter’s DOMSpider tracked xml:base attributes. Since the xml:base attributes may appear on ancestors of the XLinks rather than on the XLinks themselves, a NodeIterator really isn’t appropriate for tracking them. The key problem is that xml:base has hierarchical scope. That is, an xml:base attribute only applies to the element on which it appears and its descendants. While the filter could easily be adjusted to notice elements that have xml:base attributes as well as those that have xlink:type="simple" attributes, an iterator really can’t tell which other elements any given xml:base attribute applies to.

DOM Level 3 will add a getBaseURI() method to the Node interface that will alleviate the need to track xml:base attributes manually. In fact, this will be even more effective than the manual tracking of last chapter’s example, because it will also notice different base URIs that arise from external entities. Revising the spider() method to take advantage of this only requires changing a couple of lines of code as follows:

String wholePage = absolutize(xlink.getBaseURI(), uri);

Unfortunately, this method is not yet supported by any common parsers. However, it should be implemented in the not too distant future.

Prev | Up | Next
Chapter 12. The DOM Traversal Module | Home | TreeWalker
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 20, 2002


TreeWalker
Prev Chapter 12. The DOM Traversal Module Next

TreeWalker

The purpose of TreeWalker is much the same as that of NodeIterator, traversing a subtree of a document rooted at a particular node and filtered by both node type and custom logic. However, it differs from NodeIterator in that the traversal model is based on a tree with parents, children, and sibling nodes rather than a linear list with only previous and next nodes. Since the traversal model is very similar to what’s already available in the Node interface, tree walkers aren’t as commonly used as NodeIterator. However, the ability to filter the nodes that appear in the tree can be very useful on occasion.

Example 12.7 summarizes the TreeWalker interface. It has getter methods that return the configuration of the TreeWalker, methods to get and set the current node, and methods to move from the current node to its parent, first child, last child, previous sibling, next sibling, previous node, and next node. In all cases these methods return null if there is no such node; e.g. you ask for the last child of an empty element.

Example 12.7. The TreeWalker interface

package org.w3c.dom.traversal;

public interface TreeWalker {

  public Node       getRoot();
  public int        getWhatToShow();
  public NodeFilter getFilter();
  public boolean    getExpandEntityReferences();

  public Node       getCurrentNode();
  public void       setCurrentNode(Node currentNode)
   throws DOMException;
  public Node       parentNode();
  public Node       firstChild();
  public Node       lastChild();
  public Node       previousSibling();
  public Node       nextSibling();
  public Node       previousNode();
  public Node       nextNode();

}

A TreeWalker object is always positioned at one of the nodes in its sub-tree. It begins its existence positioned at the first node in document order. From there you can change the walker’s position by invoking nextNode(), previousNode(), parentNode(), firstChild(), lastChild(), previousSibling() and nextSibling(). In the event that there is no parent, sibling, or child relative to the current node within the walker’s tree, these methods all return null. You can find out where the walker is positioned with currentNode().

TreeWalker objects are created in almost exactly the same way as NodeIterator objects. That is, you cast the Document object you want to walk to DocumentTraversal, and invoke its createTreeWalker() method. The createTreeWalker() method takes the same four arguments with the same four meanings as createNodeIterator(): the root node of the subtree to walk, an int constant specifying which types of nodes to display, a custom NodeFilter object or null, and a boolean indicating whether or not to expand entity references.

Note

If the root node is filtered out either by whatToShow or by the NodeFilter, then the sub-tree being walked may not have a single root. In other words, it’s more like a DocumentFragment than a Document. As long as you’re cognizant of this possibility, this is not a large problem.

TreeWalkers are called for whenever the hierarchy matters; that is, whenever what’s important is not just the node itself but its parent and other ancestor nodes. For example, suppose you want to generate a list of examples in a DocBook document in the following format:

Example 1.1: A plain text document indicating an order for 12 Birdsong Clocks, SKU 244
Example 1.2: An XML document indicating an order for 12 Birdsong Clocks, SKU 244
Example 1.3: A document indicating an order for 12 Birdsong Clocks, SKU 244?
…
Example 2.1: An XML document that labels elements with schema simple types
Example 2.2: URLGrabber
Example 2.3: URLGrabberTest
…

As you saw in the last chapter, DocBook documents are structured roughly like this:

<book>
  …
  <chapter>
    …
    <example id="filename.java">
      <title>Some Java Program</title>
      <programlisting>import javax.xml.parsers;
        // more Java code…
      </programlisting>
    </example>
    …
    <example id="filename.xml">
      <title>Some XML document</title>
      <programlisting><![CDATA[<?xml version="1.0"?>
<root>
  …
</root>]]></programlisting>
     </example>
    …
  </chapter>
  more chapters…
</book>

For maximum convenience we want a TreeWalker that only sees book, chapter, example, and title elements. However, title elements should only be allowed when they represent the title of an example, not a chapter or a figure or anything else. We can set whatToShow to NodeFilter.SHOW_ELEMENT to limit the walker to elements and design a NodeFilter that only picks out these four elements. Example 12.8 demonstrates this filter.

Example 12.8. The ExampleFilter class

import org.w3c.dom.traversal.NodeFilter;
import org.w3c.dom.*;

public class ExampleFilter implements NodeFilter {

  public short acceptNode(Node node) {

    Element candidate = (Element) node;
    String name = candidate.getNodeName();
    if (name.equals("example")) return FILTER_ACCEPT;
    else if (name.equals("chapter")) return FILTER_ACCEPT;
    else if (name.equals("book")) return FILTER_ACCEPT;
    else if (name.equals("title")) {
      // Is this the title of an example, in which case we accept
      // it, or the title of something else, in which case we
      // reject it?
      Node parent = node.getParentNode();
      if ("example".equals(parent.getNodeName())) {
        return FILTER_ACCEPT;
      }
    }
    return FILTER_SKIP;

  }

}

In each case when an element is rejected, acceptNode() returns FILTER_SKIP, not FILTER_REJECT. For TreeWalker, unlike NodeIterator, the difference is important. By returning FILTER_SKIP, acceptNode() indicates that this node should not be reported but that its children should be. If acceptNode() returns FILTER_REJECT for a node, then neither that node nor any of its descendants would be covered.

The TreeWalker is simply a view of the document. It does not itself change the document or the nodes the document contains. For example, even though the ExampleFilter hides all text nodes, these can still be extracted from a title element. Example 12.9 walks the tree and pulls out these titles using this filter.

Example 12.9. Navigating a sub-tree with TreeWalker

import javax.xml.parsers.*;
import org.w3c.dom.*;
import org.w3c.dom.traversal.*;
import org.xml.sax.SAXException;
import java.io.IOException;

public class ExampleList {

  public static void printExampleTitles(Document doc) {

    // Create the NodeIterator
    DocumentTraversal traversable = (DocumentTraversal) doc;
    TreeWalker walker = traversable.createTreeWalker(
     doc.getDocumentElement(), NodeFilter.SHOW_ELEMENT,
     new ExampleFilter(), true);

    // The TreeWalker starts out positioned at the root
    Node chapter = walker.firstChild();
    int chapterNumber = 0;
    while (chapter != null) {
      chapterNumber++;
      Node example = walker.firstChild();
      int exampleNumber = 0;
      while (example != null) {
        exampleNumber++;
        Node title = walker.firstChild();
        String titleText = TextExtractor.getText(title);
        titleText = "Example " + chapterNumber + "."
         + exampleNumber + ": " + titleText;
        System.out.println(titleText);
        // Back up to the example
        walker.parentNode();
        example = walker.nextSibling();
      }
      // Reposition the walker on the parent chapter
      walker.parentNode();
      // Go to the next chapter
      chapter = walker.nextSibling();
    }

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java ExampleList URL");
      return;
    }

    String url = args[0];

    try {
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      DocumentBuilder parser = factory.newDocumentBuilder();

      // Check for the traversal module
      DOMImplementation impl = parser.getDOMImplementation();
      if (!impl.hasFeature("traversal", "2.0")) {
        System.out.println(
         "A DOM implementation that supports traversal is required."
        );
        return;
      }

      // Read the document
      Document doc = parser.parse(url);
      printExampleTitles(doc);

    }
    catch (SAXException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not check " + url
      );
    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println("Could not locate a JAXP parser");
    }

  } // end main

}

The use of TreeWalker here and NodeIterator in TextExtractor make this task a lot simpler than it otherwise would be. Hiding all the irrelevant parts means, among other things, you don’t have to worry about the complexities of example elements that appear at different depths in the tree or insignificant white space that may sporadically add extra text nodes where you don’t expect them. The traversal package enables you to boil down a document to the minimum structure relevant to your problem.

Prev | Up | Next
NodeFilter | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 19, 2002


Summary
Prev Chapter 12. The DOM Traversal Module Next

Summary

Both NodeIterator and TreeWalker can simplify the traversal of nodes of interest in a document. NodeIterator presents nodes as a one-dimensional list. TreeWalker presents them as a tree. However, otherwise the behavior and purpose of these two interfaces is much the same.

Both NodeIterator and TreeWalker traverse a subtree of the nodes in the document. Exactly which nodes belong in this subtree depends on four factors:

  • The root node of the subtree (which is often not the root node or root element of the document). Only this node and its descendants will be seen by the NodeIterator/TreeWalker.

  • whatToShow, an int constant containing bit-flags identifying which types of nodes to include. The basic bit flags for these constants are available as public final static variables in the NodeFilter interface; e.g. NodeFilter.SHOW_ELEMENT, NodeFilter.SHOW_COMMENT, NodeFilter.SHOW_TEXT, etc.

  • A NodeFilter object whose acceptNode() method returns NodeFilter.FILTER_ACCEPT, NodeFilter.FILTER_REJECT or NodeFilter.FILTER_SKIP for each node passed to it.

  • A boolean flag specifying whether or not to expand entity references.

Each of these is specified when the NodeIterator or TreeWalker is created by the createNodeIterator()/createTreeWalker() method in the DocumentTraversal interface. In a DOM implementation that supports the traversal module, all classes that implement the Document interface also implement the DocumentTraversal interface, so you can create a NodeIterator or TreeWalker over a document by casting the corresponding Document object to DocumentTraversal.

Prev | Up | Next
TreeWalker | Home | Chapter 13. Output from DOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified January 20, 2002


Chapter 13. Output from DOM
Prev Next

Chapter 13. Output from DOM

Table of Contents

Xerces Serialization
OutputFormat
DOM Level 3
Creating DOMWriters
Serialization Features
Filtering Output
Summary

Traditionally, one area in which DOM has been quite weak is serialization, the outputting of an in-memory DOM Document object into a text file. In fact, it’s even possible to use DOM to create Document objects that cannot be serialized as well-formed XML files. (For example, DOM allows an element to have attributes with the same namespace prefix but different namespace URIs.) Serialization has been left as a task for vendor specific classes such as Xerces’s XMLSerializer. However, DOM Level 3 adds several classes for writing XML documents into files, onto the network, or anything else you can hook an OutputStream to.

Xerces Serialization

The Apache XML Project’s Xerces-J includes the org.apache.xml.serialize package for writing DOM Document objects onto output streams. Although this class is bundled with Xerces, it works with any DOM Level 2 implementation. It does not depend on the details of the Xerces implementation classes, only on the standard DOM interfaces.

The basic technique for serializing documents with org.apache.xml.serialize is as follows:

  1. Configure an OutputFormat object with the serialization options you want.

  2. Connect an OutputStream to the location where you want to store the data.

  3. Use the OutputStream and the OutputFormat to construct a new XMLSerializer object.

  4. Pass the Document object you want to serialize to the XMLSerializer’s serialize() method.

For example, the following code could replace the JAXP ID transform from several examples in the last few chapters:

try {
  OutputFormat format = new OutputFormat(document);
  XMLSerializer output = new XMLSerializer(System.out, format);
  output.serialize(document);
}
catch (IOException e) {
  System.err.println(e);
}

XMLSerializer has a number of advantages over JAXP including maintaining the document type declaration`. Furthermore, it’s much more configurable.

XMLSerializer has several constructors. Which to choose depends on whether you want to write to an OutputStream or a Writer and and whether or not you want to provide an OutputFormat when you create the serializer:

public XMLSerializer();
public XMLSerializer(OutputFormat format);
public XMLSerializer(Writer out, OutputFormat format);
public XMLSerializer(OutputStream out, OutputFormat format);

Generally I recommend you specify both the format and the stream or writer when you construct the XMLSerializer. However, if you don’t want to, you can set them or change them later with these methods:

public void setOutputFormat(OutputFormat format);
public void setOutputByteStream(OutputStream out);
public void setOutputCharStream(Writer out);

You must specify either the byte stream or the char stream before you can serialize.

If you don’t want to serialize the entire document, you can pass just an Element or a DocumentFragment to the serialize() method:

public void serialize(DocumentFragment fragment)
    throws IOException;
public void serialize(Document doc)
    throws IOException;
public void serialize(Element element)
    throws IOException;

There are many other methods in the XMLSerializer class. However, mostly you should ignore these unless you’re subclassing it.

Prev | Up | Next
Summary | Home | OutputFormat
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified February 10, 2002


OutputFormat
Prev Chapter 13. Output from DOM Next

OutputFormat

The detailed behavior of a serializer is controlled by an OutputFormat object. This class can configure almost any aspect of serialization, including setting the maximum line length, changing the indenting, specifying which elements have their text escaped as CDATA sections, and more. There are even a few options that have the potential to make your documents malformed. For instance, if you add an element to the list of non-escaping elements, then any reserved characters like < and & that appear in its text content will be output as themselves rather than escaped as &lt; and &amp;.

One of the most frequent requests for serializers is pretty printing data with extra line breaks and indentation. Within reasonable limits, the OutputFormat class can provide this. Simply pass true to setIndenting(), pass the number of spaces you want each level to be indented to setIndent(), and pass the maximum line length to setLineWidth(). Example 13.1 demonstrates.

Example 13.1. Using Xerces’ OutputFormat class to pretty print XML

import java.math.*;
import java.io.IOException;
import org.w3c.dom.*;
import javax.xml.parsers.*;
import org.apache.xml.serialize.*;

public class IndentedFibonacci {

  public static void main(String[] args) {

    try {

      // Find the implementation
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setNamespaceAware(true);
      DocumentBuilder builder = factory.newDocumentBuilder();
      DOMImplementation impl = builder.getDOMImplementation();

      // Create the document
      Document doc = impl.createDocument(null,
       "Fibonacci_Numbers", null);

      // Fill the document
      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      Element root = doc.getDocumentElement();

      for (int i = 0; i < 10; i++) {
        Element number = doc.createElement("fibonacci");
        Text text = doc.createTextNode(low.toString());
        number.appendChild(text);
        root.appendChild(number);

        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }

      // Serialize the document
      OutputFormat format = new OutputFormat(doc);
      format.setLineWidth(65);
      format.setIndenting(true);
      format.setIndent(2);
      XMLSerializer serializer = new XMLSerializer(System.out, format);
      serializer.serialize(doc);

    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a JAXP factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println(
        "Could not locate a JAXP DocumentBuilder class"
      );
    }
    catch (DOMException e) {
      System.err.println(e);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

}

When run, this program produces the following output:

C:\XMLJAVA>java IndentedFibonacci
<?xml version="1.0" encoding="UTF-8"?>
<Fibonacci_Numbers>
  <fibonacci>1</fibonacci>
  <fibonacci>1</fibonacci>
  <fibonacci>2</fibonacci>
  <fibonacci>3</fibonacci>
  <fibonacci>5</fibonacci>
  <fibonacci>8</fibonacci>
  <fibonacci>13</fibonacci>
  <fibonacci>21</fibonacci>
  <fibonacci>34</fibonacci>
  <fibonacci>55</fibonacci>
</Fibonacci_Numbers>

I think you’ll agree that this looks much more attractive than the smushed together output from the bare serialization without any extra white space. One warning, however: white space is significant in XML. Adding this white space has changed the document. This is not the same document as existed before it was pretty printed. For this application the extra white space is insignificant. However, this is not true in all XML applications.

White space is just the beginning of what the OutputFormat class can control. Other features include the MIME media type, the XML declaration, the system and public IDs for the document type, which elements’ content should be escaped as CDATA sections and more. Here are the various properties you can control by invoking various methods on OutputFormat. In some cases the default is document dependent. When it’s not the default value is given in parentheses.

Method

This is normally set to one the three values xml, html or text, indicating the type of output that is desired. The serializer uses this value to configure itself. The default value is determined by the type of document being serialized.

public void setMethod(String method);
public String getMethod();
public static String whichMethod(Document doc);
Media type (null)

The MIME media type for the output such as application/xml or application/xhtml+xml. This will not be included in the document itself, but may be used as part of the stream's metadata if it's written in to a file system or onto an HTTP connection or some such.

public void setMediaType(String version);
public String getMediaType();
public static String whichMediaType(Document doc);
Version (1.0)

The version number used in the encoding declaration. This should always be "1.0". Do not change this.

public void setVersion(String version);
public String getVersion();
Standalone (no)

The value of the standalone attribute in the XML declaration. This should be true for "yes" and false for "no".

public void setStandalone(boolean standalone);
public boolean getStandalone();
Encoding (UTF-8)

The encoding specifed in the encoding attribute in the XML declaration and used to convert characters to bytes when serializing onto an OutputStream.

public void setEncoding(String encoding);
public String getEncoding();
Omit XML declaration (false)

If true, then no XML declaration is output. If false, an XML declaration is written.

public void setOmitXMLDeclaration(boolean omitXMLDeclaration);
public boolean getOmitXMLDeclaration();
Document type

This specifies the system and public IDs of the external DTD subset given in the document type declaration. These values are used only if the Document being serialized does not contain a DocumentType object of its own.

public void setDoctype(String publicID, String systemID);
public String getDoctypePublic();
public String getDoctypeSystem();
public static String whichDoctypePublic(Document doc);
public static String whichDoctypeSystem(Document doc);
Omit Document Type (false)

If true, then no document type declaration is output. If false, a document type declaration is written. If the document does not have a document type declaration and none has been set with setDoctype(), then no document type declaration will be written, regardless of the value of this property.

public void setOmitDocumentType(boolean omitDocumentType);
public boolean getDocumentType();
Non-escaping elements

The elements whose text node children should not be escaped using entity references.

public void setNonEscapingElements(String[] elementNames);
public String[] getNonEscapingElements(String[] elementNames);
public boolean isNonEscapingElement(String name);
CDATA elements

The elements whose text content should be enclosed in a CDATA section.

public void setCDATAElements(String[] elementNames);
public String[] getCDATAElements(String[] elementNames);
public boolean isCDATAElement(String name);
Omit Comments (false)

If true, then comments in the document are not written onto the output. If false, they are written.

public void setOmitComments(boolean omitComments);
public boolean getOmitComments();
Indenting (false)

If true, then the serializer will add indents at each level and wrap lines that exceed the maximum line width. If false it won't. The number of spaces to indent is set by the indent property, and the column to wrap at is set by the line width property.

public void setIndenting(boolean indenting);
public boolean getIndenting();
Indent (4)

The number of spaces to indent each level if indenting is true.

public void setIndent(int indent);
public int getIndent();
Line width (72)

The maximum number of characters in a line when indenting is true. Setting this to zero turns off line wrapping completely.

public void setLineWidth(int width);
public int getLineWidth();
Line Separator (\n)

The character or characters to use for a line break. You shouldonly set this property to a carriage return, a linefeed, or a carriage return-linefeed pair.

public void setLineSeparator(String separator);
public String getLineSeparator();

Example 13.2 uses these methods to create a valid MathML document encoded in ISO-8859-1 with a document type declaration, an XML declaration, no comments, a 65 character maximum line width, a two space indent, a standalone declaration with the value yes, and the MIME media type application/xml:

Example 13.2. Using Xerces’ OutputFormat class to pretty print MathML

import java.math.*;
import java.io.*;
import org.w3c.dom.*;
import javax.xml.parsers.*;
import org.apache.xml.serialize.*;

public class ValidFibonacciMathML {

  public static String MATHML_NS
   = "http://www.w3.org/1998/Math/MathML";

  public static void main(String[] args) {

    try {

      // Find the implementation
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setNamespaceAware(true);
      DocumentBuilder builder = factory.newDocumentBuilder();
      DOMImplementation impl = builder.getDOMImplementation();

      // Create the document
      Document doc
       = impl.createDocument(MATHML_NS, "math", null);

      // Fill the document
      BigInteger low  = BigInteger.ONE;
      BigInteger high = BigInteger.ONE;

      Element root = doc.getDocumentElement();
      root.setAttribute("xmlns", MATHML_NS);

      for (int i = 1; i <= 10; i++) {
        Element mrow = doc.createElementNS(MATHML_NS, "mrow");

        Element mi = doc.createElementNS(MATHML_NS, "mi");
        Text function = doc.createTextNode("f(" + i + ")");
        mi.appendChild(function);

        Element mo = doc.createElementNS(MATHML_NS, "mo");
        Text equals = doc.createTextNode("=");
        mo.appendChild(equals);

        Element mn = doc.createElementNS(MATHML_NS, "mn");
        Text value = doc.createTextNode(low.toString());
        mn.appendChild(value);

        mrow.appendChild(mi);
        mrow.appendChild(mo);
        mrow.appendChild(mn);

        root.appendChild(mrow);

        BigInteger temp = high;
        high = high.add(low);
        low = temp;
      }

      // Serialize the document onto System.out
      OutputFormat format = new OutputFormat(doc);
      format.setLineWidth(65);
      format.setIndenting(true);
      format.setIndent(2);
      format.setEncoding("ISO-8859-1");
      format.setDoctype("-//W3C//DTD MathML 2.0//EN",
       "http://www.w3.org/TR/MathML2/dtd/mathml2.dtd");
      format.setMediaType("application/xml");
      format.setOmitComments(true);
      format.setOmitXMLDeclaration(false);
      format.setVersion("1.0");
      format.setStandalone(true);

      XMLSerializer serializer
       = new XMLSerializer(System.out, format);
      serializer.serialize(doc);

    }
    catch (FactoryConfigurationError e) {
      System.out.println("Could not locate a JAXP factory class");
    }
    catch (ParserConfigurationException e) {
      System.out.println(
        "Could not locate a JAXP DocumentBuilder class"
      );
    }
    catch (DOMException e) {
      System.err.println(e);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

}

Here’s the beginning of the output:

C:\XMLJAVA>java ValidFibonacciMathML
D:\books\XMLJAVA\examples\13>java ValidFibonacciMathML
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE math PUBLIC "-//W3C//DTD MathML 2.0//EN"
                  "http://www.w3.org/TR/MathML2/dtd/mathml2.dtd">
<math>
  <mrow>
    <mi>f(1)</mi>
    <mo>=</mo>
    <mn>1</mn>
  </mrow>
  <mrow>
    <mi>f(2)</mi>
    <mo>=</mo>
    <mn>1</mn>
  </mrow>
…

You can imagine other requests for the serializer. For example, maybe you want a line break after each </mrow> end-tag but no line breaks inside mrow elements. OutputFormat doesn’t give you enough control to arrange serialization at this level of detail, but you could write a custom subclass of XMLSerializer that accomplishes this.

Prev | Up | Next
Chapter 13. Output from DOM | Home | DOM Level 3
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 05, 2002


DOM Level 3
Prev Chapter 13. Output from DOM Next

DOM Level 3

DOM Level 3 will finally add a standard Load and Save package so that it will be possible to write completely implementation independent DOM programs. This package, org.w3c.dom.ls, is identified by the feature strings LS-Load and LS-Save. The loading parts includes the DOMBuilder interface you've already encountered. The saving part is based on the DOMWriter interface. DOMWriter is more powerful than XMLSerializer Whereas XMLSerializer is limited to outputting documents, document fragments, and elements, DOMWriter can output any kind of node at all. Furthermore, you can install a filter into a DOMWriter that controls its output.

Caution

This section is based on very early bleeding edge technology and specifications, particularly the July 25, 2002 Working Draft of the Document Object Model (DOM) Level 3 Abstract Schemas and Load and Save Specification and Xerces-J 2.0.2. Even with Xerces-J 2.1, most of the code in this section won’t even compile, much less run. Furthermore, it’s virtually guaranteed that the details in this section will change before DOM3 becomes a final recommendation.

As shown by the method signatures in Example 13.3, DOMWriter can copy a Node object from memory into serialized bytes or characters. It has methods to write XML nodes onto a Java OutputStream or a String. The most common kind of node you’ll write is a Document, but you can write all the other kinds of node as well such as Element, Attr, and Text. This interface also has methods to control exactly how the output is formatted and how errors are reported.

Example 13.3. The DOM3 DOMWriter interface

package org.w3c.dom.ls;

public interface DOMWriter {

  public void    setFeature(String name, boolean state)
   throws DOMException;
  public boolean canSetFeature(String name, boolean state);
  public boolean getFeature(String name) throws DOMException;

  public String  getEncoding();
  public void    setEncoding(String encoding);
  public String  getNewLine();
  public void    setNewLine(String newLine);

  public boolean writeNode(OutputStream out, Node node)
   throws Exception;
  public String writeToString(Node node) throws DOMException;

  public DOMErrorHandler getErrorHandler();
  public void setErrorHandler(DOMErrorHandler errorHandler);

  public DOMWriterFilter getFilter();
  public void setFilter(DOMWriterFilter filter);

}

Note

DOMWriter is not a java.io.Writer. In fact, it even prefers OutputStreams to Writers. The name is just a coincidence.

The primary purpose of this interface is to write nodes into strings or onto streams. These nodes can be complete documents or parts thereof like elements or text nodes. For example, this code fragment uses the DOMWriter object writer to copy the Document object doc onto System.out and copy its root element into a String:

try {
  DOMWriter writer;
  // initialize the DOMWriter...
  writer.writeNode(document, System.out);
  String root = writer.writeToString(document.getDocumentElement());
}
catch (Exception e) {
  System.err.println(e);
}

DOMWriter also has several methods to configure the output. The setNewLine() method can choose the line separator used for output. The only legal values are carriage return, a line feed, or both; that is, in Java parlance, "\r", "\n", or "\r\n". You can also set this to null to indicate you want the platform’s default value.

The setEncoding() method changes the character encoding used for the output. Which encodings any given serializer supports varies from implementation to implementation, but common values include UTF-8, UTF-16, and ISO-8859-1. UTF-8 is the default if a value is not supplied. For example, this writer sets up the output for use on a Macintosh:

DOMWriter writer;
// initialize the DOMWriter...
writer.setNewLine("\r");
writer.setEncoding("MacRoman");

More detailed control of the output can be achieved by getting and setting features of the DOMWriter, as you’ll see shortly.

The setErrorHandler() method can install an org.w3c.dom.DOMErrorHandler object that receives notification of any problems that arise when outputting a node such as an element that uses the same prefix for two different namespace URIs on two attributes. This is a callback interface, similar to org.xml.sax.ErrorHandler but even simpler since it doesn’t use different methods for different kinds of errors. Example 13.4 shows this interface. The handleError() method returns true if processing should continue after the error, false if it shouldn’t.

Example 13.4. The DOM3 DOMErrorHandler interface

package org.w3c.dom;

public interface DOMErrorHandler {

  public boolean handleError(DOMError error);

}

In Xerces-2, the XMLSerializer class implements the DOMWriter interface, so if you prefer you can use these methods instead of the ones discussed in the last section. Example 13.5 demonstrates a complete program that builds a simple SVG document in memory and writes it into the file circle.svg in the current working directory using a \r\n line end and the UTF-16 encoding. The error handler is set to an anonymous inner class that prints error messages on System.err and returns false to indicate that processing should stop when an error is detected.

Example 13.5. Serializing with DOMWriter

import org.w3c.dom.*;
import org.apache.xerces.dom3.*;
import org.apache.xerces.dom3.ls.DOMWriter;
import org.apache.xml.serialize.XMLSerializer;
import java.io.IOException;
import javax.xml.parsers.*;

public class SVGCircle {

  public static void main(String[] args) {

    try {
      // Find the implementation
      DocumentBuilderFactory factory
       = DocumentBuilderFactory.newInstance();
      factory.setNamespaceAware(true);
      DocumentBuilder builder = factory.newDocumentBuilder();
      DOMImplementation impl = builder.getDOMImplementation();

      // Create the document
      DocumentType svgDOCTYPE = impl.createDocumentType(
       "svg", "-//W3C//DTD SVG 1.0//EN",
       "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"
      );
      Document doc = impl.createDocument(
       "http://www.w3.org/2000/svg", "svg", svgDOCTYPE);

      // Fill the document
      Node rootElement = doc.getDocumentElement();
      Element circle = doc.createElementNS(
       "http://www.w3.org/2000/svg", "circle");
      circle.setAttribute("r", "100");
      rootElement.appendChild(circle);

      // Serialize the document onto System.out
      DOMWriter writer = new XMLSerializer();
      writer.setNewLine("\r\n");
      writer.setEncoding("UTF-16");
      writer.setErrorHandler(
        new DOMErrorHandler() {
          public boolean handleError(DOMError error) {
            System.err.println(error.getMessage());
            return false;
          }
        }
      );
      writer.writeNode(System.out, doc);

    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

Note

Xerces-J 2.1 currently puts the DOMWriter interface in the org.apache.xerces.dom3.ls package instead of the org.w3c.dom.ls package. The Xerces team is trying to keep the experimental DOM3 classes separate from the main API until DOM3 is more stable.

Creating DOMWriters

Example 13.5 depends on Xerces-specific classes. It won’t work with GNU-JAXP or Oracle or other parsers, even after these parsers are upgraded to support DOM3. However, you can write the code in a much more parser-independent fashion by using the DOMImplementationLS interface, shown in Example 13.6, to create concrete implementations of DOMWriter, rather than constructing the implementation classes directly. DOMImplementationLS is a sub-interface of DOMImplementation that adds three methods to create new DOMBuilders, DOMWriters, and DOMInputSources.

Example 13.6. The DOM3 DOMImplementationLS interface

package org.w3c.dom.ls;

public interface DOMImplementationLS {

  public static final short MODE_SYNCHRONOUS  = 1;
  public static final short MODE_ASYNCHRONOUS = 2;

  public DOMWriter      createDOMWriter();
  public DOMInputSource createDOMInputSource();
  public DOMBuilder     createDOMBuilder(short mode,
   String schemaType) throws DOMException;

}

You retrieve a concrete instance of this factory interface by using the DOM3 DOMImplementationRegistry factory class introduced in Chapter 10 to request a DOMImplementation object that supports the LS-Save feature. Then you cast that object to DOMImplementationLS. For example,

try {
  DOMImplementation impl = DOMImplementationRegistry
   .getDOMImplementation("Core 2.0 LS-Save 3.0");
  if (impl != null) {
      DOMImplementationLS implls = (DOMImplementationLS) impl;
      DOMWriter writer = implls.createDOMWriter();
      writer.writeNode(System.out, document);
  }
  else {
    System.out.println(
     "Could not find a DOM3 Save compliant parser.");
  }
}
catch (Exception e) {
  System.err.println(e);
}

Using this technique, it’s straightforward to write a completely implementation independent program to generate and serialize XML documents. Example 13.7 demonstrates. It uses the DOMImplementationRegistry class to load the DOMImplementationLS and the DOMWriter class to output the final result. Otherwise, it just uses the standard DOM2 classes that you've seen in previous chapters.

Example 13.7. An implementation independent DOM3 program to build and serialize an XML document

import org.w3c.dom.*;
import org.w3c.dom.ls.*;

public class SVGDOMCircle {

  public static void main(String[] args) {

    try {
      // Find the implementation
      DOMImplementation impl
       = DOMImplementationRegistry.getDOMImplementation(
          "Core 2.0 LS-Load 3.0 LS-Save 3.0");
      if (impl == null) {
        System.out.println(
         "Could not find a DOM3 Load-Save compliant parser.");
        return;
      }

      // Create the document
      DocumentType svgDOCTYPE = impl.createDocumentType(
       "svg", "-//W3C//DTD SVG 1.0//EN",
       "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"
      );
      Document doc = impl.createDocument(
       "http://www.w3.org/2000/svg", "svg", svgDOCTYPE);

      // Fill the document
      Node rootElement = doc.getDocumentElement();
      Element circle = doc.createElementNS(
       "http://www.w3.org/2000/svg", "circle");
      circle.setAttribute("r", "100");
      rootElement.appendChild(circle);

      // Serialize the document onto System.out
      DOMImplementationLS implls = (DOMImplementationLS) impl;
      DOMWriter writer = implls.createDOMWriter();
      writer.writeNode(System.out, doc);

    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

This program has to test for both the LS-Load and LS-Save features because it’s not absolutely guaranteed that an implementation that has one will have the other, especially in the early days of DOM3.

Serialization Features

The defaults used by the writeNode() and writeToString() methods are acceptable for most uses. However, occasionally you want a little more control over the serialized form. For instance, you might want the output to be pretty printed with extra white space added to indent the elements nicely. Or perhaps you want the output to be in canonical form. All of this and more can be controlled by setting features in the writer before invoking the write method.

Defined features include:

normalize-characters, optional, default true

If true, output text should be normalized according to the W3C Character Model. For example, the word café would be represented as the four character string c a f é rather than the five character string c a f e combining_acute_accent. Implementations are only required to support a false value for this feature.

split-cdata-sections, required, default true

If true, CDATA sections containing the CDATA section end delimiter ]]> are split into pieces and the ]]> included in a raw text node. If false, such a CDATA section is not split. Instead an error is reported and output stops.

entities, required, default true

If true, entity references like &copy; included in the output. If false, they are not. Instead their replacement text is included.

whitespace-in-element-content, optional, default true

If true, all white space is output. If false, text nodes containing only white space are deleted if the parent element’s declaration from the DTD/schema does not allow #PCDATA to appear at that point.

discard-default-content, required, default true

If true, the implementation will attempt write out any nodes whose presence can be inferred from the DTD or schema; e.g. default attribute values. If false, it won’t include them explicitly.

canonical-form, optional, default false

If true, the document will be written according to the rules specified by the Canonical XML specification. For instance attributes will be lexically ordered and CDATA sections will not be included. If false, then the exact output is implementation dependent.

format-pretty-print, optional, default false

If true, white space will be adjusted to “pretty print” the XML. Exactly what this means, e.g. how many spaces elements are indented or what maximum line length is used, is left up to implementations.

validation, optional, default false

If true, then the document’s schema is used to validate the document as it is being output. Any validation errors that are discovered are reported to the the registered error handler. (Both validation and error handlers are other new features in DOM3.)

In addition implementations may define additional custom features. These names will generally begin with vendor specific prefixes like “apache:” or “oracle:”. For portability, you should check for the existence of such a feature with canSetFeature() before setting it. Otherwise, you’re likely to encounter an unexpected DOMException when the program is run with a different parser.

For example, this code fragment attempts to output the Document object doc onto the OutputStream out in canonical form. However, if the implementation of DOMWriter doesn’t support Canonical XML, it just outputs the document in the normal way:

try {
  DOMWriter writer = new XMLSerializer();
  if (writer.canSetFeature("canonical-form", true)) {
    writer.setFeature("canonical-form", true);
  }
  writer.writeNode(out, doc);
}
catch (Exception e) {
  System.err.println(e);
}

Filtering Output

One of the more original aspects of the DOMWriter API is the ability to attach filters to a writer that remove certain nodes from the output. A DOMWriterFilter is a sub-interface of NodeFilter from last chapter’s traversal API, and works almost exactly like it. This shouldn’t be too surprising since serializing a document is just another tree-walking operation.

To perform output filtering you first implement the DOMWriterFilter interface shown in Example 13.8. As with the NodeFilter superinterface, the acceptNode() method returns one of the three named constants NodeFilter.FILTER_ACCEPT, NodeFilter.FILTER_REJECT, or NodeFilter.FILTER_SKIP to indicate whether or not a particular node and its descendants should be output. (This method isn’t listed here because it’s inherited from the superinterface.)

Example 13.8. The DOMWriterFilter interface

package org.w3c.dom.ls;

public interface DOMWriterFilter extends NodeFilter {

  public int getWhatToShow();

}

The getWhatToShow() method returns an int constant indicating which kinds of nodes are passed to this filter for processing. This is a combination of the bit constants used by NodeIterator and TreeWalker in the last chapter; that is, NodeFilter.SHOW_ELEMENT, NodeFilter.SHOW_TEXT, NodeFilter.SHOW_COMMENT, etc.

Chapter 8 demonstrated a SAX filter that removed everything that wasn’t in the XHTML namespace from a document. Example 13.9 is a DOMWriterFilter that accomplishes the same task.

Example 13.9. Filtering everything that isn’t XHTML on output

import org.w3c.dom.*;
import org.w3c.dom.traversal.NodeFilter;
import org.w3c.dom.ls.DOMWriterFilter;

public class XHTMLFilter implements DOMWriterFilter {

  public final static String XHTML_NAMESPACE
   = "http://www.w3.org/1999/xhtml";

  // This filter only operates on elements. Everything else
  // will be output without passing through the filter. However
  // descendants of non-XHTML elements will not be output
  // because their ancestor elements have been rejected.
  // Note that this means we don't fully handle nested XHTML;
  // e.g. XHTML contains SVG which contains XHTML.
  // XHTML inside SVG will not be output.
  public int getWhatToShow() {
    return NodeFilter.SHOW_ELEMENT;
  }

  public short acceptNode(Node node) {

    // Is this necessary or does getWhatToShow() handle this????
    // I've requested clarification from the DOM working group.
    int type = node.getNodeType();
    if (type != Node.ELEMENT_NODE) {
      return NodeFilter.FILTER_ACCEPT;
    }

    String namespace = node.getNamespaceURI();
    if (XHTML_NAMESPACE.equals(namespace)) {
      return NodeFilter.FILTER_ACCEPT;
    }
    else {
     return NodeFilter.FILTER_SKIP;
    }

  }

}

The one thing this doesn’t filter out is non-XHTML attributes. Those are written out with their elements. They are not passed to acceptNode(). To filter out attributes from other namespaces would require a custom DOMWriter. You might be able to remove them from the element nodes passed to acceptNode(), but this would modify the in-memory tree as well as the streamed output. Furthermore, although Java doesn’t support this, the IDL code for DOMWriter indicates that the Node passed to acceptNode() is read-only. The underlying implementation is probably not expecting acceptNode() to modify its argument. Doing so is asking for corrupt data structures.

You can install a filter into a DOMWriter using the setFilter() method. Then any node the filter rejects will not be serialized. Example 13.10 uses the above XHTMLFilter to output pure XHTML from an input document that might contain SVG, MathML, SMIL, or other non-XHTML elements.

Example 13.10. Using a DOMWriterFilter

import org.w3c.dom.*;
import org.w3c.dom.ls.*;

public class XHTMLPurifier {

  public static void main(String[] args) {

    try {
      // Find the implementation
      DOMImplementation impl
       = DOMImplementationRegistry.getDOMImplementation(
          "Core 2.0 LS-Load 3.0 LS-Save 3.0");
      if (impl == null) {
        System.out.println(
         "Could not find a DOM3 Load-Save compliant parser.");
        return;
      }
      DOMImplementationLS implls = (DOMImplementationLS) impl;

      // Load the parser
      DOMBuilder parser = implls.createDOMBuilder(
       DOMImplementationLS.MODE_SYNCHRONOUS);

      // Parse the document
      Document doc = parser.parseURI(document);

      // Serialize the document onto System.out while filtering
      DOMWriter writer = implls.createDOMWriter();
      DOMWriterFilter filter = new XHTMLFilter();
      writer.setFilter(filter);
      writer.writeNode(System.out, doc);

    }
    catch (Exception e) {
      System.err.println(e);
    }

  }

}

Prev | Up | Next
OutputFormat | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 11, 2002


Summary
Prev Chapter 13. Output from DOM Next

Summary

Serialization is the process of taking an in-memory DOM data structure such as a Document object and writing it out onto a stream as bytes or characters. DOM Level 2 and earlier do not include any standard classes or interfaces for performing serialization. Consequently every implementation of DOM must provide its own unique API for performing this necessary task. In Xerces, that need is filled by XMLSerializer class. The OutputFormat class stores the various configuration options for an XMLSerializer including encoding, white space added for pretty printing, line separator, maximum line width, which elements to output using CDATA sections, and more.

DOM Level 3 finally adds a standard set of serialization interfaces. The DOMWriter interface provides basic methods for serializing a Document object onto a stream or into a string. The insignificant formatting of the output can be controlled by setting various features such as format-canonical and discard-default-content to either true or false. A DOMWriterFilter can control which nodes are output.

Prev | Up | Next
DOM Level 3 | Home | Chapter 14. JDOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified February 08, 2002


Chapter 14. JDOM
Prev Next

Chapter 14. JDOM

Table of Contents

What is JDOM?
Creating XML Elements with JDOM
Creating XML Documents with JDOM
Writing XML Documents with JDOM
Document Type Declarations
Namespaces
Reading XML Documents with JDOM
Navigating JDOM Trees
Talking to DOM Programs
Talking to SAX Programs
Configuring SAXBuilder
SAXOutputter
Java Integration
Serializing JDOM Objects
Synchronizing JDOM Objects
Testing Equality
Hash codes
String representations
Cloning
What JDOM doesn’t do
Summary

Tree-based APIs like DOM are very useful when programmers want to keep the entire document in memory at once with random access to the entire tree. Unfortunately DOM suffers from a number of design flaws and limitations that make it less than ideal as a Java API for processing XML. These include:

  • DOM had to be backwards compatible with the hackish, poorly thought out, unplanned object models used in third generation web browsers.

  • DOM was designed by a committee trying to reconcile differences between the object models implemented by Netscape, Microsoft, and other vendors. They needed a solution that was at least minimally acceptable to everybody, which resulted in an API that’s maximally acceptable to no one.

  • DOM is a cross-language API defined in IDL, and thus limited to those features and classes that are available in essentially all programming languages, including not fully-object oriented scripting languages like JavaScript and Visual Basic. It is a lowest common denominator API. It does not take full advantage of Java, nor does it adhere to Java best practices, naming conventions, and coding standards.

  • DOM must work for both HTML (not just XHTML, but traditional malformed HTML) and XML.

Together these constraints made DOM a lot more clumsy and hard-to-use than it should have been. I’m virtually certain that if you’ve read the last five chapters, you’ve often found yourself muttering in rather colorful language about some of the more brain damaged aspects of DOM. I know I certainly did as I wrote those chapters. In almost every case, the specific problem that elicits such complaints is a result of one of the above four constraints on the design of DOM.

JDOM is a tree-based API for processing XML documents with Java that threw out DOM’s limitations and assumptions and started from scratch. It is designed purely for XML, purely for Java, and with no concern for backwards compatibility with earlier, similar APIs. It is thus much cleaner and much simpler than DOM. Most developers find JDOM to be far more intuitive and easy to use than DOM. It’s not that JDOM will enable you to do anything you can’t do with DOM. However, writing the same program with JDOM will normally take you less time and have fewer bugs when finished, simply because of the greater intuitiveness of the API. In many ways, JDOM is to DOM as Java is to C++, a much improved, incompatible replacement for the earlier more complex technology.

Caution

JDOM is still in beta at the time of this writing. This chapter is based on the most current CVS version available, which is shortly after beta-8 and somewhere before beta-9. The API has been stabilizing and I don’t foresee any major changes between now and 1.0. However, a number of the details are likely to shift. I’d definitely check the method signatures with the latest version of the JavaDoc API documentation.

What is JDOM?

JDOM is an open source, tree-based, pure Java API for parsing, creating, manipulating, and serializing XML documents. JDOM was invented by Brett McLaughlin and Jason Hunter in the Spring of 2000. I asked Jason how it happened, and here’s what he told me:

In the early months of 2000, in a time before I knew Brett, I found myself working with XML for a contract project and growing increasingly frustrated with DOM as a way to solve my problems. My mind had an expectation for what a Java-based XML manipulation API would look like. DOM wasn’t anything like it.

In the spring of 2000, I attended Brett’s talk on DOM and SAX at the O’Reilly Conference on Enterprise Java. I was hoping he’d share with me the DOM philosophy so I could see why reality wasn’t matching my expectations. Rather than clearing things up, I found every fifth slide in his presentation was titled "Gotcha!" and listed one more thing you had to watch out for.

After his talk we sat down together on the lawn in San Jose. It was a gorgeous spring day. He was just about to release a book that was clearly destined to be a bestseller (Java and XML buzzwords in the title, what can go wrong, right elharo?). I was telling him some of what that means for a person’s career, based on my experience with a popular servlets book. I used the opportunity to ask him (someone far more expert in XML than myself at the time), "Why does it have to be like this?" He thought about it, we talked about it, and ten minutes later we decided to start an open source project to create a Java-specific XML object model. It was the first alternative to DOM in the Java world.

We worked for about a month designing the early API. We each had our role to play. Brett made sure the API was consistent with XML specifications. I made sure the API was acceptable to a Java programmer who wanted to just use XML and get on with their life. We had two private betas, then a public beta 3. James Duncan Davidson was helpful during the two private betas, especially on the interfaces vs classes debate.

Since then numerous people have contributed to JDOM’s development including Alex Rosen, Alex Chafee, James Duncan Davidson, Philip Nelson, Jools Enticknap, Bradley S. Huffman, and yours truly.

JDOM is open source like SAX and DOM. (Proprietary XML APIs really have not caught on.) Hunter and McLaughlin publish it under the very liberal Apache license. Essentially you can do anything you want with it except use the name “JDOM” for derivative works. It has already been forked once resulting in James Strachan’s dom4j.

dom4j

James Strachan forked JDOM in late 2000 to experiment with using interfaces built by factory methods to represent the nodes instead of concrete classes built by constructors. The result was dom4j.

dom4j has some features I like, including integrated XPath support and a generic Node interface that makes document navigation a lot simpler. However, in my experience most programmers find it much easier to work with class-based APIs like JDOM than with pure interface-based APIs like dom4j and DOM. Furthermore, classes can enforce constraints such as “the name property of an Element must be a legal XML name”. Interfaces can’t do that. In my opinion, dom4j makes it too easy to slip out of the constraints of XML and produce a malformed document.

Like DOM, JDOM represents an XML document as a tree composed of elements, attributes, comments, processing instructions, text nodes, CDATA sections, and so forth. The entire tree is available at any time. Unlike SAX, JDOM can access any part of the tree at any time. Unlike DOM, all the different kinds of nodes in the tree are represented by concrete classes rather than interfaces. Furthermore, there is no generic Node interface or class which all the different node classes implement or extend. (This is personally my least favorite aspect of the JDOM design. It makes tree-walking and search operations far more cumbersome than they are in DOM.)

JDOM is written in and for Java. It consistently uses the Java coding conventions and the class library. For example, all primary JDOM classes have equals(), toString(), and hashCode() methods. They all implement the Cloneable and Serializable interfaces. The children of an Element or a Document object are stored in a java.util.List. JDOM strives to be correct not just with respect to XML but also with respect to Java.

JDOM does not itself include a parser. Instead it depends on a SAX parser with a custom ContentHandler to parse documents and build JDOM models from them. Xerces 1.4.4 is bundled with JDOM. However, it can work equally well with any SAX2 compliant parser including Crimson, Ælfred, the Oracle XML Parser for Java, Piccolo, Xerces-2, and more. Any of these can read an XML document and feed it into JDOM. JDOM can also convert DOM Document objects into JDOM Document objects, which is useful for piping the output of existing DOM programs to the input of a JDOM program. However, if you’re working with a stream of XML data read from a disk or a network, it’s preferable to use SAX to produce the JDOM tree because it avoids the overhead of building the in-memory tree twice in two different representations.

Like DOM (and unlike SAX) JDOM can build a new XML tree in memory. Data for the tree can come from a non-XML source like a database, from literals in the Java program, or from calculations as in many of the Fibonacci number examples in this book. When creating new XML documents from scratch (rather than reading them from a parser), JDOM checks all the data for well-formedness. For example, unlike many DOM implementations, JDOM does not allow programs to create comments whose data includes the double hyphen -- or elements and attributes whose namespace mapping conflict in impossible ways.

Once a document has been loaded into memory, whether by creating it from scratch or by parsing it from a stream, JDOM can modify the document. A JDOM tree is fully read-write. All parts of the tree can be moved, deleted, and added to, subject to the usual restrictions of XML. (For instance, you can add an attribute to an element but not to a comment.) Unlike DOM, there are no annoying read-only sections of the tree that you can’t change.

Finally, when you’re finished working with a document in memory, JDOM lets you serialize it back out to disk or onto a stream as a sequence of bytes. JDOM provides numerous options to specify the encoding, indenting, line end characters, and other details of serialization. Alternately, if you don’t want to convert the document to a stream, you can produce a SAX event sequence or a DOM document as output instead.

Prev | Up | Next
Summary | Home | Creating XML Elements with JDOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 12, 2002


Creating XML Elements with JDOM
Prev Chapter 14. JDOM Next

Creating XML Elements with JDOM

One of my favorite things about JDOM is that 90% of the time, it works exactly like I expect it to work. I don’t have to look at the documentation because my best guess is almost always right. This is only sometimes true for SAX and almost never true for DOM. For example, suppose you wanted to create the JDOM representation of this element:

<fibonacci/>

The simplest, most obvious solution I can imagine is this:

Element element = new Element("fibonacci");

Guess what? That’s exactly how you do create an element in JDOM. Compare that to the DOM approach:

DocumentBuilderFactory factory
 = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
DOMImplementation impl = builder.getDOMImplementation();
Document doc = impl.createDocument(
 null, "Fibonacci_Numbers", null);
Element element = doc.createElement("fibonacci");

I think any sane developer would agree that JDOM is simpler, at least for this initial example.

Now suppose you want to give the element some text content like this:

<fibonacci>8</fibonacci>

Here’s first thing that occurs to me:

Element element = new Element("fibonacci");
element.setText("8");

Guess what? That works. It is exactly how JDOM fills an element with text. Unlike DOM you don’t have to create any separate text nodes. JDOM does use a Text class internally, but you don’t need to know about that. You can just use strings.

Now suppose you want to add an attribute to the element like this:

<fibonacci index="6">8</fibonacci>

The first thing that occurs to me is this:

Element element = new Element("fibonacci");
element.setText("8");
element.setAttribute("index", "6");

Once again, the most obvious solution is the correct solution. This is a running theme in JDOM. Try the simplest thing that could possibly work, and chances are it will work.

Note

On the other hand, there are some limits to simplicity, mostly established by XML. JDOM strives to be as simple as it can be and no simpler. It does not hide the genuinely complex parts of XML. For instance, it faithfully reproduces all the complexity of XML namespaces, including the parts that have occasionally been labeled “psychotic”. For example, it allows the same prefix to map to different URIs in different parts of the document. This compares favorably with some other APIs (I’m thinking of ElectricXML here) that pretend XML is simpler than it really is. In general, if something is complex in JDOM, it’s because that same something is equally complex in XML.

Of course many elements contain child elements, comments, processing instructions, and other things besides pure text. In this case you can attach them by calling the addContent() method. For example, suppose you want to create this element:

<sequence>
  <number>3</number>
  <number>5</number>
</sequence>

First you need to create three Element objects, two for number elements and one for the sequence element. Then you need to add the number elements to the sequence element in the order you want them to appear. For example,

Element element = new Element("sequence");
Element firstNumber = new Element("number");
Element secondNumber = new Element("number");
firstNumber.setText("3");
secondNumber.setText("5");
element.addContent(firstNumber);
element.addContent(secondNumber);

Actually I’ve cheated a bit here. What this really produces is this element:

<sequence><number>3</number><number>5</number></sequence>

White space is significant in XML and thus significant in JDOM. If you want the nicely indented element, you also need to add some strings containing the appropriate white space like this:

Element element = new Element("sequence");
Element firstNumber = new Element("number");
Element secondNumber = new Element("number");
firstNumber.setText("3");
secondNumber.setText("5");
element.addContent("\n  ");
element.addContent(firstNumber);
element.addContent("\n  ");
element.addContent(secondNumber);
element.addContent("\n");

If you only care about the extra white space when the document is serialized, you can ask an XMLOutputter to insert it for you. I’ll cover this soon.

Prev | Up | Next
Chapter 14. JDOM | Home | Creating XML Documents with JDOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 04, 2002


Creating XML Documents with JDOM
Prev Chapter 14. JDOM Next

Creating XML Documents with JDOM

Let’s begin with a simple JDOM program that creates this XML document:

<?xml version="1.0"?>
<GREETING>Hello JDOM!</GREETING>

Since all documents should have root elements, we’ll need to create the root GREETING element first, then use that element to create the document:

  Element root = new Element("GREETING");
  root.setText("Hello JDOM!");
  Document doc = new Document(root);

Note

Initially the Element object is not associated with any Document. It is free-standing. This contrasts with DOM where all nodes are always part of some document. JDOM allows nodes to stand on their own if that’s useful. However, JDOM does not allow a node to be part of two documents at once. Before an Element can be transferred into a new Document it must first be detached from its old document using its detach() method.

You can reorder the method calls. For example, you might want to modify the root element after it’s been attached to the document like this:

  Element root = new Element("GREETING");
  Document doc = new Document(root);
  root.setText("Hello JDOM!");

You can even create the Document object first using a no-args constructor and then set its root element like this:

  Document doc = new Document();
  Element root = new Element("GREETING");
  root.setText("Hello JDOM!");
  doc.setRootElement(root);

In this case the document begins its life in a temporarily malformed state, and any attempt to do almost anything with it except set a root element or add content to the document’s prolog will fail with an java.lang.IllegalStateException.

Prev | Up | Next
Creating XML Elements with JDOM | Home | Writing XML Documents with JDOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified April 18, 2002


Writing XML Documents with JDOM
Prev Chapter 14. JDOM Next

Writing XML Documents with JDOM

Once you’ve created a document, you’re likely to want to serialize it to a network socket, a file, a string, or some other stream. JDOM’s org.jdom.output.XMLOutputter class does this in a standard way. You can create an XMLOutputter object with a no-args constructor and then write a document onto an OutputStream with its output() method. For example, this code fragment writes the Document object named doc onto System.out.

    XMLOutputter outputter = new XMLOutputter();
    try {
      outputter.output(doc, System.out);
    }
    catch (IOException e) {
      System.err.println(e);
    }

Besides streams you can also output a document onto a java.io.Writer. However, it’s recommended that you use an OutputStream because it’s generally not possible to determine the underlying encoding of a Writer and set the encoding declaration accordingly.

Besides documents, XMLOutputter can write elements, attributes, CDATA sections, and all the other JDOM node classes. For example, this code fragment writes an empty element named Greeting onto System.out:

    XMLOutputter outputter = new XMLOutputter();
    try {
      Element element = new Element("Greeting");
      outputter.output(element, System.out);
    }
    catch (IOException e) {
      System.err.println(e);
    }

This may occasionally be useful; but if you write anything other than a single Document or Element onto a stream, the result probably won’t be a well-formed XML document.

Finally, instead of writing onto a stream or writer, you can use the outputString() methods to store an XML document or node in a String. This is often useful when passing XML data through non-XML aware systems. For example, this code fragment stores an empty element named Greeting in the String variable named hello:

  XMLOutputter outputter = new XMLOutputter();
  Element element = new Element("Greeting");
  String hello = outputter.outputString(element);

Example 14.1 puts this all together with a simple program that generates the Fibonacci series in XML format.

Example 14.1. A JDOM program that produces an XML document containing Fibonacci numbers

import org.jdom.*;
import org.jdom.output.XMLOutputter;
import java.math.BigInteger;
import java.io.IOException;

public class FibonacciJDOM {

  public static void main(String[] args) {

    Element root = new Element("Fibonacci_Numbers");

    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;

    for (int i = 1; i <= 5; i++) {
      Element fibonacci = new Element("fibonacci");
      fibonacci.setAttribute("index", String.valueOf(i));
      fibonacci.setText(low.toString());
      root.addContent(fibonacci);

      BigInteger temp = high;
      high = high.add(low);
      low = temp;
    }

    Document doc = new Document(root);
    // serialize it onto System.out
    try {
      XMLOutputter serializer = new XMLOutputter();
      serializer.output(doc, System.out);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

}

The output is as follows:

D:\books\XMLJAVA\examples\14>java FibonacciJDOM
<?xml version="1.0" encoding="UTF-8"?>
<Fibonacci_Numbers><fibonacci index="1">1</fibonacci><fibonacci
index="2">1</fibonacci><fibonacci index="3">2</fibonacci>
<fibonacci index="4">3</fibonacci><fibonacci index="5">5
</fibonacci></Fibonacci_Numbers>

This isn’t especially pretty. There are a couple of ways to clean it up. First off you can recognize that white space is significant in XML and by default JDOM faithfully reproduces it. Thus if you want the output to be indented, you could add strings containing line breaks and extra space in the right place. However, if you happen to know that white space is not significant in the particular XML vocabulary the program writes, then you can ask the XMLOutputter to format the document for you. For example, this XMLOutputter inserts the default line ending after elements and indents elements by two spaces per each layer of the hierarchy:

      XMLOutputter serializer = new XMLOutputter();
      serializer.setIndent("  "); // use two space indent
      serializer.setNewlines(true);
      serializer.output(doc, System.out);

Now the output looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<Fibonacci_Numbers>
  <fibonacci index="1">1</fibonacci>
  <fibonacci index="2">1</fibonacci>
  <fibonacci index="3">2</fibonacci>
  <fibonacci index="4">3</fibonacci>
  <fibonacci index="5">5</fibonacci>
</Fibonacci_Numbers>

Much prettier, I think you’ll agree.

You can also specify the amount of indenting to use and whether or not to add line breaks as arguments to the XMLOutputter() constructor like this:

      XMLOutputter serializer = new XMLOutputter("  ", true);
      serializer.output(doc, System.out);

For another example, let’s revisit FlatXMLBudget, Example 4.2 from Chapter 4. Recall that its purpose was to read a tab-delimited file containing financial data and convert it into XML. The method that actually generated the XML was convert(), and it did this by writing strings onto an OutputStream like so:

  public static void convert(List data, OutputStream out)
   throws IOException {

    Writer wout = new OutputStreamWriter(out, "UTF8");
    wout.write("<?xml version=\"1.0\"?>\r\n");
    wout.write("<Budget>\r\n");

    Iterator records = data.iterator();
    while (records.hasNext()) {
      wout.write("  <LineItem>\r\n");
      Map record = (Map) records.next();
      Set fields = record.entrySet();
      Iterator entries = fields.iterator();
      while (entries.hasNext()) {
        Map.Entry entry = (Map.Entry) entries.next();
        String name = (String) entry.getKey();
        String value = (String) entry.getValue();
        // some of the values contain ampersands and less than
        // signs that must be escaped
        value = escapeText(value);

        wout.write("    <" + name + ">");
        wout.write(value);
        wout.write("</" + name + ">\r\n");
      }
      wout.write("  </LineItem>\r\n");
    }
    wout.write("</Budget>\r\n");
    wout.flush();

  }

JDOM can make this method quite a bit simpler as well as eliminating the need for the escapeText() method completely, since JDOM handles that internally:

  public static void convert(List data, OutputStream out)
   throws IOException {

    Element budget = new Element("Budget");

    Iterator records = data.iterator();
    while (records.hasNext()) {
      Element lineItem = new Element("LineItem");
      budget.addContent(lineItem);

      Map record = (Map) records.next();
      Set fields = record.entrySet();
      Iterator entries = fields.iterator();
      while (entries.hasNext()) {
        Map.Entry entry = (Map.Entry) entries.next();
        String name = (String) entry.getKey();
        String value = (String) entry.getValue();

        Element category = new Element(name);
        category.setText(value);
        lineItem.addContent(category);
      }
    }

    Document doc = new Document(budget);
    XMLOutputter outputter = new XMLOutputter("  ", true);
    outputter.output(doc, out);
    out.flush();

  }

The disadvantage to this approach is that even though the input is streamed the output is not. The entire document is built and stored in memory before the first byte of output is written. This can be a problem in memory limited devices or with large documents.

Prev | Up | Next
Creating XML Documents with JDOM | Home | Document Type Declarations
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified April 19, 2002


Document Type Declarations
Prev Chapter 14. JDOM Next

Document Type Declarations

Documents created with JDOM can have document type declarations and thus can be valid. JDOM does not offer a complete object model for DTDs. However, it does allow you to point at an existing DTD or add an internal DTD subset to your documents.

Example 14.2 is a simple document type definition for the Fibonacci number documents we’ve been generating.

Example 14.2. A Fibonacci DTD

<!ELEMENT Fibonacci_Numbers (fibonacci*)>
<!ELEMENT fibonacci (#PCDATA)>
<!ATTLIST fibonacci index CDATA #IMPLIED>

Let assume that this is available at the relative URL fibonacci.dtd. Thus the following DOCTYPE declaration would make the document valid:

<!DOCTYPE Fibonacci_Numbers SYSTEM "fibonacci.dtd">

In JDOM, the DocType class represents document type declarations. You can create this object using a constructor that receives the root element name and system ID as arguments. For example,

DocType type = new DocType("Fibonacci_Numbers", "fibonacci.dtd");

You can either pass this DocType object as the second argument to the Document constructor or invoke the Document class’s setDocType() method.

Example 14.3 demonstrates a program that produces a completely valid document. However, JDOM does not provide any direct means to test the validity of a document, short of serializing it and passing the resulting stream through a validating parser.

Example 14.3. A JDOM program that produces an XML document containing Fibonacci numbers

import org.jdom.*;
import org.jdom.output.XMLOutputter;
import java.math.BigInteger;
import java.io.IOException;

public class ValidFibonacci {

  public static void main(String[] args) {

    Element root = new Element("Fibonacci_Numbers");
    DocType type = new DocType("Fibonacci_Numbers", "fibonacci.dtd");
    Document doc = new Document(root, type);

    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;

    for (int i = 1; i <= 5; i++) {
      Element fibonacci = new Element("fibonacci");
      fibonacci.setAttribute("index", String.valueOf(i));
      fibonacci.setText(low.toString());
      root.addContent(fibonacci);

      BigInteger temp = high;
      high = high.add(low);
      low = temp;
    }

    // serialize with two space indents and extra line breaks
    try {
      XMLOutputter serializer = new XMLOutputter("  ", true);
      serializer.output(doc, System.out);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

}

Here’s the output with the document type declaration in place:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Fibonacci_Numbers SYSTEM "fibonacci.dtd">
<Fibonacci_Numbers>
  <fibonacci index="1">1</fibonacci>
  <fibonacci index="2">1</fibonacci>
  <fibonacci index="3">2</fibonacci>
  <fibonacci index="4">3</fibonacci>
  <fibonacci index="5">5</fibonacci>
</Fibonacci_Numbers>

You can also specify a public ID for the external DTD subset. For example, if the public ID for Example 14.2 is -//Elliotte Rusty Harold//Fibonacci Example//EN, then the DocType object could be initialized like this:

DocType type = new DocType("Fibonacci_Numbers",
  "-//Elliotte Rusty Harold//Fibonacci Example//EN",
  "fibonacci.dtd");

You can also use the setInternalSubset() method to provide an internal DTD subset. As with all internal DTD subsets, this can be instead of or in addition to the external DTD subset identified by the public ID and the system ID. For example, this code fragment uses an internal DTD subset instead of an external DTD subset.

    Element root = new Element("Fibonacci_Numbers");

    DocType type = new DocType("Fibonacci_Numbers");
    String dtd = "<!ELEMENT Fibonacci_Numbers (fibonacci*)>\n";
    dtd += "<!ELEMENT fibonacci (#PCDATA)>\n";
    dtd += "<!ATTLIST fibonacci index CDATA #IMPLIED>\n";
    type.setInternalSubset(dtd);

    Document doc = new Document(root, type);

The document produced includes an internal DTD subset and no system or public ID:

<?xml version="1.0" encoding="UTF-8"
<!DOCTYPE Fibonacci_Numbers [
<!ELEMENT Fibonacci_Numbers (fibonacci*)>
<!ELEMENT fibonacci (#PCDATA)>
<!ATTLIST fibonacci index CDATA #IMPLIED>
]>
<Fibonacci_Numbers>
  <fibonacci index="1">1</fibonacci>
  …

Other programs and documents might specify both an internal DTD subset and a system and public ID for the external DTD subset.

Note

DTDs and document type declarations are mostly important for serialization. If the document is written onto a stream and read by some other program, then that other program may take advantage of the DTD for validation or application of default attribute values and so forth. However, JDOM itself doesn’t pay a lot of attention to the DTD. In fact, as far as it’s concerned the various parts are all just strings. It does not, for example, apply any default attribute values indicated by either the internal or external DTD subsets. If the initial JDOM Document object is created by a validating parser (rather than directly in memory), then that parser will report all the default attribute values the same as specified attribute values. However, changes made to the Document’s DocType after the document is initially parsed will not have any affect on the content of the rest of the Document.

Prev | Up | Next
Writing XML Documents with JDOM | Home | Namespaces
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 29, 2002


Namespaces
Prev Chapter 14. JDOM Next

Namespaces

Suppose instead of the simple custom vocabulary I’ve been using so far, you wanted to use the standard MathML presentation vocabulary as shown in Example 14.4.

Example 14.4. A MathML document containing the first three Fibonacci numbers

<?xml version="1.0"?>
<mathml:math xmlns:mathml="http://www.w3.org/1998/Math/MathML">
  <mathml:mrow>
    <mathml:mi>f(1)</mathml:mi>
    <mathml:mo>=</mathml:mo>
    <mathml:mn>1</mathml:mn>
  </mathml:mrow>
  <mathml:mrow>
    <mathml:mi>f(2)</mathml:mi>
    <mathml:mo>=</mathml:mo>
    <mathml:mn>1</mathml:mn>
  </mathml:mrow>
  <mathml:mrow>
    <mathml:mi>f(3)</mathml:mi>
    <mathml:mo>=</mathml:mo>
    <mathml:mn>2</mathml:mn>
  </mathml:mrow>
</mathml:math>

The biggest change from the previous examples is that MathML uses namespaces on all the elements. The basic JDOM rule about namespaces is that when an element or attribute is in a namespace, rather than specifying its full qualified name, you give its local name, its prefix, and its URI, in that order. If the element is in the default namespace, omit the prefix. You do not need to add attributes for the namespace declarations. The outputter will figure out reasonable places to put them when the document is serialized.

For example, the following statement creates the root mathml:math element:

Element root = new Element("math",
                           "mathml",
                           "http://www.w3.org/1998/Math/MathML");

Example 14.5 demonstrates a complete program that generates MathML from JDOM. The namespace has to be specified on each element. Being a child of an element in the http://www.w3.org/1998/Math/MathML namespace is not sufficient (or necessary) to make the element part of the http://www.w3.org/1998/Math/MathML namespace. Each element (and each attribute) has its own namespace URI which is independent of the other namespace URIs in the document.

Example 14.5. A JDOM program that uses namespaces

import org.jdom.Element;
import org.jdom.Document;
import org.jdom.output.XMLOutputter;
import java.math.BigInteger;
import java.io.IOException;

public class PrefixedFibonacci {

  public static void main(String[] args) {

    Element root = new Element("math", "mathml",
     "http://www.w3.org/1998/Math/MathML");

    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;

    for (int i = 1; i <= 5; i++) {

      Element mrow = new Element("mrow", "mathml",
       "http://www.w3.org/1998/Math/MathML");

      Element mi = new Element("mi", "mathml",
       "http://www.w3.org/1998/Math/MathML");
      mi.setText("f(" + i + ")");
      mrow.addContent(mi);

      Element mo = new Element("mo", "mathml",
       "http://www.w3.org/1998/Math/MathML");
      mo.setText("=");
      mrow.addContent(mo);

      Element mn = new Element("mn", "mathml",
       "http://www.w3.org/1998/Math/MathML");
      mn.setText(low.toString());
      mrow.addContent(mn);

      BigInteger temp = high;
      high = high.add(low);
      low = temp;
      root.addContent(mrow);

    }

    Document doc = new Document(root);
    try {
      XMLOutputter serializer = new XMLOutputter("  ", true);
      serializer.output(doc, System.out);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

}

Using the default namespace is even easier. Simply specify the namespace on each element but omit the prefix. Example 14.6 demonstrates.

Example 14.6. A JDOM program that uses the default namespace

import org.jdom.Element;
import org.jdom.Document;
import org.jdom.output.XMLOutputter;
import java.math.BigInteger;
import java.io.IOException;

public class UnprefixedFibonacci {

  public static void main(String[] args) {

    Element root = new Element("math", "mathml",
     "http://www.w3.org/1998/Math/MathML");

    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;

    for (int i = 1; i <= 5; i++) {

      Element mrow = new Element("mrow",
       "http://www.w3.org/1998/Math/MathML");

      Element mi = new Element("mi",
       "http://www.w3.org/1998/Math/MathML");
      mi.setText("f(" + i + ")");
      mrow.addContent(mi);

      Element mo = new Element("mo",
       "http://www.w3.org/1998/Math/MathML");
      mo.setText("=");
      mrow.addContent(mo);

      Element mn = new Element("mn",
       "http://www.w3.org/1998/Math/MathML");
      mn.setText(low.toString());
      mrow.addContent(mn);

      BigInteger temp = high;
      high = high.add(low);
      low = temp;
      root.addContent(mrow);

    }

    Document doc = new Document(root);
    try {
      XMLOutputter serializer = new XMLOutputter("  ", true);
      serializer.output(doc, System.out);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  }

}

Although these examples only use a single namespace, you are by no means limited to a single namespace per document in JDOM, any more than you are in XML. Each element can have whatever namespace it requires.

Prev | Up | Next
Document Type Declarations | Home | Reading XML Documents with JDOM
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified April 17, 2002


Reading XML Documents with JDOM
Prev Chapter 14. JDOM Next

Reading XML Documents with JDOM

Naturally, JDOM can read existing XML documents from files, network sockets, strings, or anything else you can hook a stream or reader to. JDOM does not, however, include its own native parser. Instead it relies on any of a number of very fast, well-tested SAX2 parsers such as Xerces and Crimson.

The rough outline for working with an existing XML document using JDOM is as follows:

  1. Construct an org.jdom.input.SAXBuilder object using a simple no-args constructor

  2. Invoke the builder’s build() method to build a Document object from a Reader, InputStream, URL, File, or a String containing a system ID.

  3. If there’s a problem reading the document, an IOException is thrown. If there’s a problem building the document, a JDOMException is thrown.

  4. Otherwise, navigate the document using the methods of the Document class, the Element class, and the other JDOM classes.

The SAXBuilder class represents the underlying XML parser. Parsing a document from a URL is straightforward. Just create a SAXBuilder object with the no-args constructor and pass the string form of the URL to its build() method. This returns a JDOM Document object. For example,

 SAXBuilder parser = new SAXBuilder();
 Document doc = parser.build("http://www.cafeconleche.org/");
 // work with the document...

That’s all there is to it. If you prefer, you can build the Document from a java.io.File, a java.net.URL, a java.io.InputStream, a java.io.Reader, or an org.xml.sax.InputSource.

The build() method throws an IOException if an I/O error such as a broken socket prevents the document from being completely read. It throws a JDOMException if the document is malformed. This is the generic superclass for most things that can go wrong while working with JDOM other than I/O errors. Example 14.7 demonstrates a simple program that checks XML documents for well-formedness by looking for these exceptions.

Example 14.7. A JDOM program that checks XML documents for well-formedness

import org.jdom.JDOMException;
import org.jdom.input.SAXBuilder;
import java.io.IOException;

public class JDOMChecker {

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println("Usage: java JDOMChecker URL");
      return;
    }

    SAXBuilder builder = new SAXBuilder();

    // command line should offer URIs or file names
    try {
      builder.build(args[0]);
      // If there are no well-formedness errors,
      // then no exception is thrown
      System.out.println(args[0] + " is well-formed.");
    }
    // indicates a well-formedness error
    catch (JDOMException e) {
      System.out.println(args[0] + " is not well-formed.");
      System.out.println(e.getMessage());
    }
    catch (IOException e) {
      System.out.println("Could not check " + args[0]);
      System.out.println(" because " + e.getMessage());
    }

  }

}

I used this program to test my Cafe con Leche web site for well-formedness. It’s supposed to be well-formed XML, but I’m often sloppy. The results were informative:

D:\books\XMLJAVA\examples\14>java JDOMChecker http://www.cafeconleche.org/
http://www.cafeconleche.org is not well formed.
Error on line 351 of document http://www.cafeconleche.org: The
element type "img" must be terminated by the matching end-tag
"</img>".

I fixed the problem. However, JDOM only reports the first error in a document, so it’s not surprising that running the program again uncovered a second problem:

D:\books\XMLJAVA\examples\14>java JDOMChecker http://www.cafeconleche.org/
http://www.cafeconleche.org is not well formed.
Error on line 363 of document http://www.cafeconleche.org: The
element type "input" must be terminated by the matching end-tag
"</input>".

Several more problems were encountered in order. Once I fixed the last one, everything finally checked out:

D:\books\XMLJAVA\examples\14>java JDOMChecker http://www.cafeconleche.org/
http://www.cafeconleche.org is well formed.

Exactly which SAX parser JDOM uses to build documents depends on the local environment. By default, JDOM relies on JAXP to choose the parser class. If that fails, it picks Xerces. However, if you really care which parser is used, specify the fully package qualified name of the XMLReader class you want as the first argument to the constructor. For example, this sets the parser as Crimson:

 SAXBuilder parser
  = new SAXBuilder("org.apache.crimson.parser.XMLReaderImpl");
 Document doc = parser.build("http://www.cafeconleche.org/");
 // work with the document...

By default, SAXBuilder only checks documents for well-formedness, not validity. If you want to validate as well, then pass the boolean true to the SAXBuilder() constructor. Then any validity errors will also cause JDOMExceptions. Example 14.8 demonstrates with a simple validation program.

Example 14.8. A JDOM program that validates XML documents

import org.jdom.JDOMException;
import org.jdom.input.SAXBuilder;
import java.io.IOException;

public class JDOMValidator {

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println("Usage: java JDOMValidator URL");
      return;
    }

    SAXBuilder builder = new SAXBuilder(true);
                                    //  ^^^^
                                    // Turn on validation

    // command line should offer URIs or file names
    try {
      builder.build(args[0]);
      // If there are no well-formedness or validity errors,
      // then no exception is thrown.
      System.out.println(args[0] + " is valid.");
    }
    // indicates a well-formedness or validity error
    catch (JDOMException e) {
      System.out.println(args[0] + " is not valid.");
      System.out.println(e.getMessage());
    }
    catch (IOException e) {
      System.out.println("Could not check " + args[0]);
      System.out.println(" because " + e.getMessage());
    }

  }

}

Here are the results from running this program across two documents, the first invalid (because it doesn’t even have a DTD) and the second valid:

D:\books\XMLJAVA\examples\14>java JDOMValidator http://cafeconleche.org/
http://cafeconleche.org is not valid.
Error on line 1 of document http://cafeconleche.org: Document root
element "html", must match DOCTYPE root "null".
D:\books\XMLJAVA\examples\14>java JDOMValidator
http://www.w3.org/TR/2000/REC-xml-20001006.html
http://www.w3.org/TR/2000/REC-xml-20001006.html is well formed.

This does assume that the default parser JDOM picks can in fact validate, which is true of most modern parsers you’re likely to encounter. However, if you really want to make sure you could always ask for a known validating parser by name. For example, this requests the Xerces SAXParser:

SAXBuilder parser
 = new SAXBuilder("org.apache.xerces.parsers.SAXParser", true);

Note

JDOM does not currently distinguish between validity and well-formedness errors. I’m working on a patch for this. Of course, any malformed document is de facto invalid.

Prev | Up | Next
Namespaces | Home | Navigating JDOM Trees
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 29, 2002


Navigating JDOM Trees
Prev Chapter 14. JDOM Next

Navigating JDOM Trees

Once you’ve parsed a document and formed a Document object, you’ll probably want to search it to select out those parts of it your program is interested in. In JDOM, most navigation takes place through the methods of the Element class. The complete children of each Element are available as a java.util.List returned by the getContent() method. Just the child elements of each Element are available in a java.util.List returned by the getChildren() method. (Yes, the terminology is a little confusing here. This is a case where JDOM is marching out of step with the rest of the XML world. JDOM uses the word children to refer only to child elements.)

Because JDOM uses the Java Collections API to manage the tree, it is simultaneously too polymorphic (everything’s an object and must be cast to the right type before you can use it) and not polymorphic enough (there’s no useful generic interface or superclass for navigation such as DOM’s Node class.) Consequently, you’re going to find yourself doing numerous tests with instanceof and casting to the determined type. This is far and away my least favorite part of JDOM’s design. Furthermore, there’s no standard traversal API as there is in DOM to help you avoid reinventing the wheel every time you need to walk a tree or iterate a document. There is a Filter interface that can simplify some of the polymorphism and casting issues a little, but it still won’t let you walk more than one level down the tree at a time.

Let’s begin with Example 14.9, a simple program that reads a document and prints the names of the elements in that document, nicely indented to show the hierarchy. Pay special attention to the listChildren() method. This recursive method is the key to the whole program.

Example 14.9. A JDOM program that lists the elements used in a document

import org.jdom.*;
import org.jdom.input.SAXBuilder;
import java.io.IOException;
import java.util.*;

public class ElementLister {

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println("Usage: java ElementLister URL");
      return;
    }

    SAXBuilder builder = new SAXBuilder();

    try {
      Document doc = builder.build(args[0]);
      Element root = doc.getRootElement();
      listChildren(root, 0);
    }
    // indicates a well-formedness error
    catch (JDOMException e) {
      System.out.println(args[0] + " is not well-formed.");
      System.out.println(e.getMessage());
    }
    catch (IOException e) {
      System.out.println(e);
    }

  }

  public static void listChildren(Element current, int depth) {

    printSpaces(depth);
    System.out.println(current.getName());
    List children = current.getChildren();
    Iterator iterator = children.iterator();
    while (iterator.hasNext()) {
      Element child = (Element) iterator.next();
      listChildren(child, depth+1);
    }

  }

  private static void printSpaces(int n) {

    for (int i = 0; i < n; i++) {
      System.out.print(' ');
    }

  }

}

The main() method simply parses a document and passes its root element to the listChildren() method along with a depth of zero. The listChildren() method indents a number of spaces equal to the depth in the hierarchy. Then it prints the name of the current element. Next it asks for a list of the children of that element by invoking getChildren(). This returns a java.util.List from the Java Collections API. This list is live. That is, any changes you make to it will be reflected in the original Element. However, this program does not take advantage of that. Instead, it retrieves a java.util.Iterator object using the iterator() method. Then it iterates through the list. Since each item in the list is known to be a JDOM Element object, each item returned by next() can be safely cast to Element and passed recursively to listChildren(). Other than the knowledge that each object in the list is an Element, every step is just standard list iteration from the Java Collections API. Internally, JDOM is actually using a special package-private subclass of List, org.jdom.ContentList; but you don’t need to know this. Everything you need to do can be done through the documented java.util.List interface.

Here’s the beginning of the output when this program is run across this chapter’s source code:

D:\books\XMLJAVA\examples\14>java ElementLister file://D/books/XMLJava/jdom.xml
chapter
 title
 caution
  para
 para
 itemizedlist
  listitem
   para
  listitem
   para
  listitem
   para
  listitem
   para
 para
 para
 caution
  para
 sect1
  title
  para
  blockquote
  …

The getChildren() method only returns elements. It misses everything else completely. For instance, it doesn’t report comments, processing instructions, or text nodes. To get this material you need to use the getContent() method which returns everything. However, this makes life a little trickier because you can no longer assume that everything in the list returned is an Element. You’ll probably need to use a big tree of if (o instance of Element) { … } else if (o instanceof Text) { … in order to choose the processing to perform on each member of the list. Example 14.10 demonstrates with a simple program that recursively lists all the nodes used in the document. Elements are identified by their name. All other items are identified just by their types.

Example 14.10. A JDOM program that lists the nodes used in a document

import org.jdom.*;
import org.jdom.input.SAXBuilder;
import java.io.IOException;
import java.util.*;

public class NodeLister {

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println("Usage: java NodeLister URL");
      return;
    }

    SAXBuilder builder = new SAXBuilder();

    try {
      Document doc = builder.build(args[0]);
      listNodes(doc, 0);
    }
    // indicates a well-formedness error
    catch (JDOMException e) {
      System.out.println(args[0] + " is not well-formed.");
      System.out.println(e.getMessage());
    }
    catch (IOException e) {
      System.out.println(e);
    }

  }

  public static void listNodes(Object o, int depth) {

    printSpaces(depth);

    if (o instanceof Element) {
      Element element = (Element) o;
      System.out.println("Element: " + element.getName());
      List children = element.getContent();
      Iterator iterator = children.iterator();
      while (iterator.hasNext()) {
        Object child = iterator.next();
        listNodes(child, depth+1);
      }
    }
    else if (o instanceof Document) {
      System.out.println("Document");
      Document doc = (Document) o;
      List children = doc.getContent();
      Iterator iterator = children.iterator();
      while (iterator.hasNext()) {
        Object child = iterator.next();
        listNodes(child, depth+1);
      }
    }
    else if (o instanceof Comment) {
      System.out.println("Comment");
    }
    else if (o instanceof CDATA) {
      System.out.println("CDATA section");
      // CDATA is a subclass of Text so this test must come
      // before the test for Text.
    }
    else if (o instanceof Text) {
      System.out.println("Text");
    }
    else if (o instanceof EntityRef) {
      System.out.println("Entity reference");
    }
    else if (o instanceof ProcessingInstruction) {
      System.out.println("Processing Instruction");
    }
    else {  // This really shouldn't happen
      System.out.println("Unexpected type: " + o.getClass());
    }

  }

  private static void printSpaces(int n) {

    for (int i = 0; i < n; i++) {
      System.out.print(' ');
    }

  }

}

Here’s the beginning of the output when this program is run across this chapter’s source code:

D:\books\XMLJAVA\examples\14>java NodeLister file://D/books/XMLJava/jdom.xml
Document
 Element: chapter
  Text
  Element: title
   Text
  Text
  Element: caution
   Text
   Element: para
    Text
   Text
  Text
  Element: para
  …

The only pieces that are missing here are the attributes and namespaces associated with each element. These are not included by either getContent() or getChildren(). If you want them, you have to ask for them explicitly using the getAttributes(), getNamespace(), getAdditionalNamespaces(), and related methods of the Element class.

In the next chapter, we’ll look more closely at the classes of objects that appear when you’re navigating a JDOM tree (Element, Text, Comment, etc.) and what you can learn from each one.

Prev | Up | Next
Reading XML Documents with JDOM | Home | Talking to DOM Programs
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 04, 2002


Talking to DOM Programs
Prev Chapter 14. JDOM Next

Talking to DOM Programs

JDOM is not an acronym. It does not stand for “Java Document Object Model”. JDOM is not directly compatible with DOM (which is an acronym). That is to say, a JDOM Element is not a DOM Element. The JDOM Element class does not implement the DOM Element interface. JDOM’s Element class has methods that the DOM Element interface does not have and vice versa. You cannot pass a JDOM Element to a method that expects a DOM Element or a DOM Element to a method that expects a JDOM Element. The same is true for the JDOM Document class and the DOM Document interface, the JDOM Attribute class and the DOM Attr interface, the JDOM ProcessingInstruction class and the DOM ProcessingInstruction interface, and so forth.

That being said, JDOM does allow you to convert JDOM documents to and from DOM documents. I don’t recommend this for new projects (which should pick one API or the other and stick with it); but it is useful for integrating new JDOM programs with legacy DOM code and vice versa.

If you already have a DOM Document object, then the org.jdom.input.DOMBuilder class can use it to generate the JDOM equivalent. The syntax is straightforward. Use the no-args DOMBuilder() to create a DOMBuilder object and then pass the DOM Document object to its build() method. For example, assuming that the variable domDocument points to an object of type org.w3c.dom.Document, this code fragment build an org.jdom.Document object from it:

DOMBuilder builder = new DOMBuilder();
org.jdom.Document jdomDocument = builder.build(domDocument);
// work with the JDOM document…

The original DOM object is not changed in any way. Furthermore, changes to the JDOM Document do not affect the DOM Document it was built from. In the other direction, future changes to the DOM Document do not affect the JDOM Document.

Moving in the other direction, from JDOM to DOM, the org.jdom.output.DOMOutputter class produces DOM Document objects from JDOM Document objects. Since this isn’t really serialization, but rather than just converting from one model to another, there aren’t nearly as many options to set as with XMLOutputter. For instance, you can’t add extra white space or select the encoding. (You could always do that later with a DOM serializer of some kind, if necessary.) For example,

DOMOutputter converter = new DOMOutputter();
org.w3c.dom.Document domDocument = converter.output(jdomDocument);
// work with the DOM document…

Once again, these documents are not connected after the initial creation of one from the other. Changes to one are not reflected in the other.

Prev | Up | Next
Navigating JDOM Trees | Home | Talking to SAX Programs
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified April 18, 2002


Talking to SAX Programs
Prev Chapter 14. JDOM Next

Talking to SAX Programs

JDOM works very well with SAX parsers. SAX is an almost ideal event model for building a JDOM tree; and when the tree is complete, JDOM makes it easy to walk the tree, firing off SAX events as you go. Since SAX is so fast and memory-efficient, SAX doesn’t add a lot of extra overhead to JDOM programs.

Configuring SAXBuilder

When reading a file or stream through a SAX parser, you can set various properties on the parser including the ErrorHandler, DTDHandler, EntityResolver, and any custom features or properties that are supported by the underlying SAX XMLReader. SAXBuilder includes several methods that just delegate these configurations to the underlying XMLReader:

public void setErrorHandler(ErrorHandler errorHandler);
public void setEntityResolver(EntityResolver entityResolver);
public void setDTDHandler(DTDHandler dtdHandler);
public void setIgnoringElementContentWhitespace(boolean ignoreWhitespace);
public void setFeature(String name, boolean value);
public void setProperty(String name, Object value);

For example, suppose you want to schema validate documents before using them. This requires three additional steps beyond the norm:

  1. Explicitly pick a parser class than is known to be able to schema validate such as org.apache.xerces.parsers.SAXParser (Most parsers can’t schema validate.)

  2. Install a SAX ErrorHandler that reports validity errors.

  3. Set the SAX feature that turns on schema validation to true. Which feature this is depends on which parser you picked in step 1. In Xerces, it’s http://apache.org/xml/features/validation/schema and you also need to turn validation on using the standard SAX feature http://xml.org/sax/features/validation.

Example 14.11 is a simple JDOM program that schema validates a URL named on the command line using Xerces. This is similar to the earlier JDOMValidator example. However here because the installed ErrorHandler (BestSAXChecker from Chapter 7) merely prints validity error messages on System.out and does not throw an exception, validity errors do not terminate the parse. The Document object is still built as long as it’s well-formed, whether or not it’s valid. You could, of course, change this behavior by using a more draconian ErrorHandler that did throw exceptions for validity errors.

Example 14.11. A JDOM program that schema validates documents

import org.jdom.JDOMException;
import org.jdom.input.SAXBuilder;
import java.io.IOException;

public class JDOMSchemaValidator {

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println("Usage: java JDOMSchemaValidator URL");
      return;
    }

    SAXBuilder builder = new SAXBuilder(
     "org.apache.xerces.parsers.SAXParser");
    builder.setValidation(true);
    builder.setErrorHandler(new BestSAXChecker());
                             // ^^^^^^^^^^^^^^
                             // From Chapter 7
    // turn on schema support
    builder.setFeature(
      "http://apache.org/xml/features/validation/schema", true);

    // command line should offer URIs or file names
    try {
      builder.build(args[0]);
    }
    // indicates a well-formedness error
    catch (JDOMException e) {
      System.out.println(args[0] + " is not well-formed.");
      System.out.println(e.getMessage());
    }
    catch (IOException e) {
      System.out.println("Could not check " + args[0]);
      System.out.println(" because " + e.getMessage());
    }

  }

}

Here’s the result from when I used this program to check a mildly invalid document. One error was reported:

D:\books\XMLJAVA\examples\14>java JDOMSchemaValidator original_hotcop.xml
Error: cvc-type.3.1.3: The value '6:20' of element 'LENGTH' is
 not valid.
 at line 10, column 24
 in entity file:///D:/books/XMLJAVA/examples/14/original_hotcop.xml

Caution

You should only use setFeature() and setProperty() for non-standard features and properties like http://apache.org/xml/features/validation/schema. SAXBuilder requires certain settings of the standard features such as http://xml.org/sax/features/namespace-prefixes and standard properties such as http://xml.org/sax/properties/lexical-handler in order to work properly. If you change these, the document may not be built correctly.

Another interesting possibility is the option to set a SAX filter that is applied to the document as it’s read:

public void setXMLFilter(XMLFilter filter);

If you use this, the JDOM Document will include only the filtered content.

SAXOutputter

Besides reading a file or stream through a SAX parser, you can also feed a JDOM document into a SAX ContentHandler using the org.jdom.output.SAXOutputter class. This class is initially configured with a ContentHandler and optionally an ErrorHandler, DTDHandler, EntityResolver, and/or LexicalHandler. The output() method walks the tree, firing off events to these handlers as it does so.

For example, suppose you’ve built a document in memory that happens to contain some XInclude elements and you’d like to resolve them. JDOM does not have any built-in support for XInclude. To JDOM, an XInclude element is just an element that happens to have the local name include and the namespace URI http://www.w3.org/2001/XInclude. However, GNU JAXP does include a filter that can resolve XIncludes. Unfortunately it’s a SAX filter rather than a JDOM filter. Not to worry. It’s straightforward to feed a JDOM document into the GNU JAXP gnu.xml.pipeline.XIncludeFilter using a SAXOutputter as shown in Example 14.12:

Example 14.12. A JDOM program that passes documents to a SAX ContentHandler

import org.jdom.*;
import org.jdom.input.SAXBuilder;
import org.jdom.output.SAXOutputter;
import java.io.IOException;
import gnu.xml.pipeline.*;
import org.xml.sax.SAXException;

public class XIncluder {

  public static void main(String[] args) {

    if (args.length == 0) {
      System.out.println("Usage: java XIncluder URL");
      return;
    }

    SAXBuilder builder = new SAXBuilder(
     "gnu.xml.aelfred2.XmlReader");

    // command line should offer URIs or file names
    try {
      Document doc = builder.build(args[0]);
      XIncludeFilter filter = new XIncludeFilter(
        new TextConsumer(System.out)
      );
      SAXOutputter outputter = new SAXOutputter(filter);
      outputter.setContentHandler(filter);
      outputter.setDTDHandler(filter);
      outputter.setLexicalHandler(filter);
      outputter.output(doc);
    }
    // indicates a well-formedness error
    catch (JDOMException e) {
      System.out.println(args[0] + " is not well-formed.");
      System.out.println(e.getMessage());
    }
    catch (SAXException e) {
      System.out.println(e.getMessage());
    }
    catch (IOException e) {
      System.out.println("Could not merge " + args[0]);
      System.out.println(" because " + e.getMessage());
    }

  }

}

Here the XIncludeFilter is itself hooked up to another GNU JAXP class, TextConsumer, which merely prints the document on a specified OutputStream.

Prev | Up | Next
Talking to DOM Programs | Home | Java Integration
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified April 18, 2002


Java Integration
Prev Chapter 14. JDOM Next

Java Integration

JDOM is the most Java-centric of all the major APIs for processing XML. The JDOM developers have given a lot of thought to exactly how the JDOM classes fit into common Java systems such as the Java Collections API, Remote Method Invocation (RMI), and the I/O framework. Unlike DOM, the behavior of JDOM objects is very well defined with respect to Java operations such as cloning and serializing.

Serializing JDOM Objects

All the major JDOM classes such as Element, Document, Namespace implement the java.io.Serializable interface. This means JDOM objects can be passed between machines using remote method invocation (RMI) and stored in files using ObjectOutputStream and ObjectInputStream.

Of course just because something is possible doesn't mean it's a good idea. XML is itself a serialization format for JDOM objects, and it's a much more broadly supported one. What's not as broadly known is that pure textual XML is also generally much faster and smaller than Java's binary object serialization. There's very little reason to use serialized JDOM objects instead of passing genuine XML documents back and forth.

At the time of this writing, the long-term plan for JDOM serialization is still under discussion. While you can use object serialization to pass JDOM objects from one virtual machine to another, both of which have the same version of JDOM, using anything beyond that is up in the air. In particular, it is highly doubtful that an object you serialize today will be able to be deserialized tomorrow in a different version of JDOM. For long-term persistence, you should absolutely use XML documents instead of serialized objects.

Synchronizing JDOM Objects

For the most part, except for a few accidental exceptions, JDOM classes are not thread-safe. You cannot use a JDOM Document, Element, or other object safely in multiple threads simultaneously unless you synchronize it properly. If you build the object in one thread, and therefter only read from in different threads (in essence, if you treat the object as if it were immutable) then you may be OK. But if you plan to write to or modify the object, you're going to have to synchronize your objects.

Testing Equality

All the core JDOM classes (Element, Attribute, ProcessingInstruction, etc.) implement the equals() method. In all cases, the test is for object identity. In other words, element1.equals(element2) if and only if element1 == element2. That is, element1 is the same element as element2.

The reasoning is that order and position are significant in XML documents. Thus two nodes can’t be equal unless they are in fact the same node. For example, consider the two Price elements in this XML fragment:

  <Item>
    <Name>2002 Toyota Camry</Name>
    <Price>$10,000</Price>
  </Item>
  <Item>
     <Name>1976 AMC Gremlin</Name>
    <Price>$10,000</Price>
  </Item>

The two Price elements are character-for-character identical. However, one is a very good price and one is a very bad price because of what their positions in the document.

This behavior is enforced by implementing the equals() method with an == test and making the method final. A typical JDOM equals() method is defined thusly:

  public final boolean equals(Object o) {
    return (this == o);
  }

Hash codes

Because JDOM tests for equality based on object identity, the default hashCode() implementation inherited from java.lang.Object suffices. However, to prevent subclasses from violating this contract, the hashCode() method is implemented as a final method that calls super.hashCode(). That is, it looks like this:

 public final int hashCode() {
    return super.hashCode();
  }

Thus it cannot be overridden, and subclasses cannot change its behavior.

String representations

The JDOM toString() methods produce strings that look like these:

[Document: No DOCTYPE declaration, Root is [Element: <html
[Namespace: http://www.w3.org/1999/xhtml]/>]]
[Element: <html [Namespace: http://www.w3.org/1999/xhtml]/>]
[Attribute: xml:lang="en"]
[Text:
]
[Attribute: type="text/css"]
[Attribute: rel="stylesheet"]
[Text: Latest Version: ]
[Element: <a [Namespace: http://www.w3.org/1999/xhtml]/>]
[Attribute: href="http://www.rddl.org/"]
[Text: June 16, 2002]

They are suitable for debugging, but not for display to an end user. In particular, they are not the serialized form of the XML node the object represents. To serialize a JDOM object onto a stream or into a String, use an XMLOutputter chained to a StringWriter. Do not use toString().

Cloning

All the core JDOM classes implement Cloneable. Except for Namespace objects (which are immutable), all clones are deep clones. For example, cloning an element makes a copy of the element's contents as well. The clone does not have a parent, and can be inserted into the same document or a different document.

Prev | Up | Next
Talking to SAX Programs | Home | What JDOM doesn’t do
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified August 23, 2002


What JDOM doesn’t do
Prev Chapter 14. JDOM Next

What JDOM doesn’t do

There is more to JDOM than what you’ve seen in the brief tour in this chapter. The next chapter will cover the capabilities of the various JDOM classes like Element and Attribute in depth, as well as taking a deeper look at how these constructs are represented. However, before we take that up, it’s worth taking a few minutes to note the features JDOM doesn’t have, and that might lead you to choose a different API instead.

First of all, JDOM cannot handle documents larger than available memory. JDOM loads the entire document tree into memory and keeps it there. JDOM is more memory efficient than some DOM implementations and less efficient than others. However, it’s never going to be competitive with SAX or XMLPULL for truly humongous documents.

Secondly, like both DOM and SAX, JDOM presents a logical model of XML documents. It does not track every last byte of the documents it parses. For instance, it cannot tell which physical entities any element came from. It treats the document as a logical whole. It cannot tell whether a character was input literally or with a character reference (a limitation it shares with DOM and SAX). It does not know the original character encoding of the document. In short, it cannot guarantee byte-for-byte faithful round trips. Merely parsing a document with JDOM and then immediately writing it back out again, may create a subtly different document, though it should still contain the same basic information.

Third, JDOM is purely about the instance document. It does not provide any real model of the DTD or schema. It cannot tell you what the type of an element is, or what its valid children are. It cannot check a Document object against a schema or a DTD.

Finally, JDOM does not currently provide any equivalent to the traversal package in DOM. This isn’t a huge problem, since writing your own tree-walking code does not require a great deal of effort, as you’ll see in the next chapter. However, a standard way of doing this would be helpful.

Prev | Up | Next
Java Integration | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 27, 2002


Summary
Prev Chapter 14. JDOM Next

Summary

JDOM is a pure Java API for processing XML documents. It is more complete than either SAX (which doesn’t offer any standard way to write new XML documents) or DOM (which can manipulate XML documents but doesn’t know how to parse or serialize them). It is also much easier to use than either SAX or DOM for most tasks. It has the convenience of a pull-based tree API with DOM and the familiarity of following standard Java conventions with SAX. However, JDOM is not SAX and it is not DOM. JDOM can transfer data to and from SAX and DOM, but it is its own API, complete unto itself.

JDOM uses concrete classes rather than interfaces. This means you can create instances of most of the node types just by passing an argument or two to a constructor. (The notable exception is the Namespace class which uses a factory method in order to implement the flyweight design pattern.) For instance, to create a new Element object for a number element, you simply type

Element element = new Element("number");

Node objects do not need to be attached to any document. However, an object can’t be part of more than one document at a time.

JDOM Document objects can also be created by parsing an existing file. This is done by the SAXBuilder class which relies on a SAX2 parser such as Xerces. JDOM Document objects can also be built from existing DOM Document objects through the DOMBuilder class. Moving in the other direction, the XMLOutputter class can serialize a JDOM Document object onto a stream. The SAXOutputter class can feed a JDOM Document into a SAX ContentHandler, and the DOMOutputter class can convert a JDOM Document into a DOM Document.

Prev | Up | Next
What JDOM doesn’t do | Home | Chapter 15. The JDOM Model
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified April 19, 2002


Chapter 15. The JDOM Model
Prev Next

Chapter 15. The JDOM Model

Table of Contents

The Document Class
The Element Class
Constructors
Navigation and Search
Attributes
The Attribute Class
The Text Class
The CDATA Class
The ProcessingInstruction Class
The Comment Class
Namespaces
The DocType class
The EntityRef Class
Summary

JDOM documents are composed of ten basic classes representing the different kinds of nodes in an XML document:

  • Document

  • Element

  • Attribute

  • Text

  • ProcessingInstruction

  • Namespace

  • Comment

  • DocType

  • EntityRef

  • CDATA

In JDOM, node objects can belong to a Document; or they can exist independently. A complete JDOM Document contains a list of its children, which normally includes Comments, ProcessingInstructions, possibly a single DocType, and one Element object for the root element. Most of the rest of the structure of the document is built from Element objects, each of which contains a list of its contents, as well as separate lists of attributes and namespaces for that element.

The Document Class

JDOM’s Document class, summarized in Example 15.1, represents a complete well-formed XML document. As a node, it is the root of the document tree. It enforces most of the rules XML imposes on documents. For instance, it is not possible to assign two DocType objects to a single Document object because an XML document cannot have two document type declarations. The single exception to this rule is that it is possible for a Document to be temporarily rootless if it’s built by the no-args constructor. However, in that case, almost anything you do with the document other than setting a root element will throw an IllegalStateException, so this is just a temporary convenience, not a loophole that allows you to break the rules of XML.

Example 15.1. The JDOM Document class

package org.jdom;

public class Document implements Serializable, Cloneable {

  protected ContentList content;
  protected DocType     docType;

  public Document()
  public Document(Element root, DocType docType)
  public Document(Element root)
  public Document(List newContent, DocType docType)
  public Document(List content)

  public boolean  hasRootElement()
  public Element  getRootElement()
  public Document setRootElement(Element rootElement)
  public Element  detachRootElement()

  public DocType  getDocType()
  public Document setDocType(DocType docType)

  public Document addContent(ProcessingInstruction pi)
  public Document addContent(Comment comment)
  public List     getContent()
  public List     getContent(Filter filter)
  public Document setContent(List newContent)
  public boolean  removeContent(ProcessingInstruction pi)
  public boolean  removeContent(Comment comment)

  // Java utility methods
  public       String  toString()
  public final boolean equals(Object o)
  public final int     hashCode()
  public       Object  clone()

}

As you can probably guess from the setter and getter methods, each Document object has the following three properties:

  • The root Element

  • A DocType object representing the document type declaration

  • A List containing the root Element and any processing instructions or comments in the prolog and epilog, in document order.

The first two may occasionally be null. A document that does not have a document type declaration will have a null DocType. The root Element may also be null, but only temporarily. You can’t do much of anything to a Document until you’ve set its root Element.

Most of the time all you do with a Document is get the root element. From that point forward, you work exclusively with the root element and its descendants until you're ready to serialize the Document. For example,

Document doc = builder.build("file.xml");
Element root = doc.getRootElement();
// work with root…

Occasionally, you may need to hunt for a particular processing instruction in the prolog or epilog. For instance, if you were writing a browser, then you would want to look for an xml-stylesheet processing instruction in the prolog by iterating through the list returned by getContent() like so:

      List content = doc.getContent();
      Iterator iterator = content.iterator();
      while (iterator.hasNext()) {
        Object next = iterator.next();
        if (next instanceof Element) {
          // This is the root element. Thus the prolog is
          // finished.
          break;
        }
        else if (next instanceof ProcessingInstruction) {
          ProcessingInstruction candidate
           = (ProcessingInstruction) next;
          if (candidate.getTarget().equals("xml-stylesheet")) {
            // Load stylesheet…
          }
        }
      }…

However, it’s uncommon to pay that much attention to the contents of the prolog (and even rarer to pay attention to the epilog). Consequently, since the addContent(), setContent(), getContent(), and removeContent() methods work pretty much the same for Document as they do for Element, I’m going to defer further discussion of these to the next section.

Prev | Up | Next
Summary | Home | The Element Class
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 05, 2002


The Element Class
Prev Chapter 15. The JDOM Model Next

The Element Class

The structure of an XML document is based on its elements, so it should come as no surprise that the Element class is one of the larger and more important in JDOM. Since JDOM does not have any generic Node class or interface, the Element class is the primary means by which a program navigates the tree to find particular content.

Each Element object has the following seven basic properties:

Local name

A String that is initialized when the Element is constructed, and which can never be null or the empty string. It is accessible through the setName() and getName() methods:

public Element setName(String name)
    throws IllegalNameException;
public String getName();
Namespace

A Namespace object which encapsulates both the namespace URI and an optional prefix. This can be the named constant Namespace.NO_NAMESPACE if the element does not have a namespace. A namespace is always set when the Element is constructed but can be changed by setNamespace(). It can be read by the getNamespace() method.

public Element setNamespace(Namespace namespace);
public String getNamespace();
Content

A List with no duplicates containing all the element’s children in order. This is accessible through the getContent() and setContent() methods. The list is live so you can change the contents of the Element using the methods of the List class.

public Element setContent(List list)
    throws IllegalAddException;
public List getContent();

In addition, individual nodes can be added to and removed from the list via the addContent() and removeContent() methods.

Parent

The parent Element that contains this Element. It will be null if this is the root element, and may be null if this Element is not currently part of a Document. This is accessible through the getParent() method:

public Element getParent();

The parent can be changed only by adding the Element to a new parent using the parent’s addContent() method. This is only possible if the Element does not already have a parent. Before a parent can adopt a child Element, the child’s detach() method must be invoked to remove it from its current parent:

public Element detach();
Owner document

The Document that contains this Element. It will be null if this Element is not currently part of a Document. It can be read by the getDocument() method:

public Document getDocument();

It can be changed by adding the Element to a new document after first detaching it from its previous parent with the detach() method.

Attributes

A List containing Attribute objects, one for each of the element’s attributes. Although JDOM stores attributes in a list for convenience, order is not significant, and is not likely to be the same as the order in which the attributes appeared in the original document. The list is accessible through the getAttributes() and setAttributes() methods:

public Element setAttributes(List attributes)
    throws IllegalAddException;
public List getAttributes();

The items in this list can be read and modified via the getAttribute(), getAttributeValue(), and setAttribute() methods. Attributes that declare namespaces are not included in this list.

Additional namespaces

A List containing Namespace objects, one for each additional namespace prefix declared by the element (that is, other than those that declare the namespace of the element and the namespaces of its attributes). As with the list of attributes, order is not significant. The entire list is accessible through the getAdditionalNamespaces() method:

public List getAdditionalNamespaces();

Namespaces can be added to and removed from the list using the addNamespaceDeclaration(), and removeNamespaceDeclaration() methods:

public Element addNamespaceDeclaration(Namespace namespace);
public Element removeNamespaceDeclaration(Namespace namespace);

In addition, there are some other properties which are not independent of the above seven. For instance, the prefix, namespace URI, and fully qualified name are separately readable through the getNamespaceURI(), getNamespacePrefix(), and getQualifiedName() convenience methods:

public String getNamespaceURI();
public String getNamespacePrefix();
public String getQualifiedName();

These just return the relevant parts of the element’s namespace and name.

All these getter methods behave pretty much like any other getter methods. That is, they return an object of the relevant type, generally a String, and do not throw any exceptions. The setter methods are more unusual, however. This is one of the few areas where JDOM does not follow standard Java conventions. Instead of returning void, these methods all return the Element object that invoked the method. That is, a.setFoo(b) returns a. Many other methods you'd naturally expect to return void also do this. The purpose is to allow setters to be chained. For example, this code fragment can build up an entire channel element in just a couple of statements:

 Element channel = (new Element("channel"))
 .addContent((new Element("title")).setText("Cafe con Leche"))
 .addContent((new Element("link"))
  .setText("http://www.cafeconleche.org/"))
 .addContent((new Element("description"))
  .setText("XML News"));

Caution

I must say that I personally don’t find this style of code easier to write or read than the multi-statement approach. However, this is why the adder and setter methods all return the object that did the adding or setting so I felt compelled to show it to you. But I really recommend strongly that you don’t use it.

Constructors

The four public Element constructors all require you to specify a local name as a String. If the element is in a namespace, then you also need to specify the namespace URI as a String or a Namespace object. The prefix can also be specified as a String or a piece of a Namespace object.

public Element(String localName)
    throws IllegalNameException;
public Element(String localName, Namespace namespace)
    throws IllegalNameException;
public Element(String localName, String namespaceURI)
    throws IllegalNameException;
public Element(String localName, String prefix, String namespaceURI)
    throws IllegalNameException;

For example, this code fragment creates four Element objects using the various constructors:

Element xmlRPCRoot = new Element("methodCall");
Element xhtmlRoot = new Element("html",
 "http://www.w3.org/1999/xhtml");
Element soapRoot = new Element("Envelope", "SOAP-ENV",
 "http://schemas.xmlsoap.org/soap/envelope/");
Namespace xsd = Namespace.getNamespace("xsd",
 "http://www.w3.org/2001/XMLSchema");
Element schemaRoot = new Element("schema", xsd);

Navigation and Search

As you learned in the last chapter, the getContent() method is the fundamental means of navigating through an XML document with JDOM. This method returns a live List which includes all the children of an element including comments, processing instructions, text nodes, and elements. To search deeper, you apply getContent() to the child elements of the current element, normally through recursion.

For example, here’s a simple program that walks the XML document tree, starting at the root element, and prints out the content of the various properties of each element. This is not the most interesting program in the book, but it does demonstrate all the major getter methods and basic navigation: Pay special attention to the process() method. You have to write a method very much like this for any JDOM program that needs to search an entire XML document. It begins with an Element (normally the root element) and recursively applies itself to all the child elements of the root element. The instanceof operator tests each object in the Element’s content list to determine its type and dispatch it to the right method. Here, TreePrinter dispatches Element objects to the process() method recursively, and ignores all other objects.

Example 15.2. Inspecting elements

import org.jdom.*;
import org.jdom.input.SAXBuilder;
import java.io.IOException;
import java.util.*;

public class TreePrinter {

  // Recursively descend the tree
  public static void process(Element element) {

    inspect(element);
    List content = element.getContent();
    Iterator iterator = content.iterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      if (o instanceof Element) {
        Element child = (Element) o;
        process(child);
      }
    }

  }

  // Print the properties of each element
  public static void inspect(Element element) {

    if (!element.isRootElement()) {
      // Print a blank line to separate it from the previous
      // element.
      System.out.println();
    }

    String qualifiedName = element.getQualifiedName();
    System.out.println(qualifiedName + ":");

    Namespace namespace = element.getNamespace();
    if (namespace != Namespace.NO_NAMESPACE) {
      String localName = element.getName();
      String uri = element.getNamespaceURI();
      String prefix = element.getNamespacePrefix();
      System.out.println("  Local name: " + localName);
      System.out.println("  Namespace URI: " + uri);
      if (!"".equals(prefix)) {
        System.out.println("  Namespace prefix: " + prefix);
      }
    }
    List attributes = element.getAttributes();
    if (!attributes.isEmpty()) {
      Iterator iterator = attributes.iterator();
      while (iterator.hasNext()) {
        Attribute attribute = (Attribute) iterator.next();
        String name = attribute.getName();
        String value = attribute.getValue();
        Namespace attributeNamespace = attribute.getNamespace();
        if (attributeNamespace == Namespace.NO_NAMESPACE) {
          System.out.println("  " + name + "=\"" + value + "\"");
        }
        else {
          String prefix = attributeNamespace.getPrefix();
          System.out.println(
           "  " + prefix + ":" + name + "=\"" + value + "\"");
        }
      }
    }

    List namespaces = element.getAdditionalNamespaces();
    if (!namespaces.isEmpty()) {
      Iterator iterator = namespaces.iterator();
      while (iterator.hasNext()) {
        Namespace additional = (Namespace) iterator.next();
        String uri = additional.getURI();
        String prefix = additional.getPrefix();
          System.out.println(
           "  xmlns:" + prefix + "=\"" + uri + "\"");
      }
    }

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java TreePrinter URL");
      return;
    }

    String url = args[0];

    try {
      SAXBuilder parser = new SAXBuilder();

      // Parse the document
      Document document = parser.build(url);

      // Process the root element
      process(document.getRootElement());

    }
    catch (JDOMException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not encode " + url
      );
    }

  } // end main

}

Here’s the beginning of output when this chapter’s XML source code is fed into TreePrinter. DocBook doesn’t use namespaces, but the XInclude elements do. The root element has some attributes, but most of the structure is based on element name alone.

D:\books\XMLJAVA>java TreePrinter jdom_model.xml
chapter:
  revision="20020430"
  status="rough"
  id="ch_jdom_model"
  xmlns:xinclude="http://www.w3.org/2001/XInclude"

title:

para:

para:

itemizedlist:

listitem:

para:

classname:
…

While in theory you could navigate and query a document using only the List objects returned by getContent(), JDOM provides many methods to simplify the process for special cases including methods that return lists containing child elements only, methods that return particular named child elements, methods that return the complete text of an element, methods that return the text of a child element, methods to remove children identified by name and reference, methods that return the first child of an element, and more.

Child Elements

The Element class has two methods (five total when you count overloaded variants separately) that only operate on the child elements of an element, and not on other content like processing intructions and text nodes. These are getChildren() and removeChildren():

public List getChildren();
public List getChildren(String name);
public List getChildren(String name, Namespace namespace);
public List removeChildren(String name);
public List removeChildren(String name, Namespace namespace);

These methods are similar to getContent() and removeContent() except that the lists returned only contain child elements, never other kinds of children like comments and processing instructions. [1] The getChildren() methods simply ignore non-elements. For instance, the earlier TreePrinter example only considered elements. Consequently, it could use the getChildren() method instead of getContent():

  public static void process(Element element) {

    inspect(element);
    List content = element.getChildren();
    Iterator iterator = content.iterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      Element child = (Element) o;
      process(child);
    }

  }

This eliminates one instanceof check and one if block. This is not a huge savings, I admit; but the code is marginally more readable. However, because JDOM uses Java’s Object-based List class, you still have to cast all the items in the list getChildren() returns to Element.

The removeChildren() methods remove all the elements that match the specified name and namespace URI. If no namespace URI is given, then it removes elements with the given name in no namespace. Other content—comments, processing instructions, text, etc.—is not touched.

For example, this method recursively descends through an element, cutting out all the note elements.

  public static void cutNotes(Element element) {

    List notes = element.getChildren("note");
    element.removeChildren(notes);
    // The element's children have changed so we have to call
    // getChildren() again
    List children = element.getChildren();
    Iterator iterator = children.iterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      Element child = (Element) o;
      cutNotes(child);
    }

  }

It’s important to remember that when an element is removed, the entire element is removed, not just its start and end-tags. Any content inside the element is lost including, in this case, elements that aren’t named note.

Single children

Often you want to follow a very specific path through a document. For instance, consider the XML-RPC request document in Example 15.3. A program that reads this is probably primarily concerned with the content of the string element.

Example 15.3. An XML-RPC request document

<?xml version="1.0"?>
<methodCall>
  <methodName>getQuote</methodName>
  <params>
    <param>
      <value><string>RHAT</string></value>
    </param>
  </params>
</methodCall>

To get the string element, you’ll ask for the string child element of the value child element of the param child element of the params child element of the root element. Rather than iterating through a list of all the child elements when there’s only one of each of these, you can ask for the one you want directly using one of the getChild() methods:

public Element getChild(String name);
public Element getChild(String name, Namespace namespace);

For example,

Element root   = document.getRootElement();
Element params = root.getChild("params");
Element param  = params.getChild("param");
Element value  = params.getChild("value");
Element symbol = params.getChild("string");

Or, more concisely,

Element symbol = document.getRootElement()
                  .getChild("params")
                  .getChild("param");
                  .getChild("value")
                  .getChild("string");

This method has two nasty problems. The first is that it only returns the first such child. If there’s more than one child element with the specified name and namespace, you still only get the first one. The second problem is that if there's no such child, then getChild() returns null, thus leading to a NullPointerException. Since both of these are very real possibilities in many applications, including XML-RPC, you should normally prefer getChildren() unless you’ve used some form of schema or DTD to verify that there’s exactly one of each child you address with these methods. getChildren() always returns a non-null list you can safely iterate through to process anywhere from zero to thousands of child elements.

Similarly, you can remove a single named child element with one of the two removeChild() methods, each of which returns the removed Element in case you want to save it for later use:

public Element removeChild(String name);
public Element removeChild(String name, Namespace namespace);

The removeChild() method shares with getChild() the problem of operating only on the first such element. However, after you’ve removed the first child, the second child is now the first. After you’ve removed that one, the original third child is now the first, and so on. Thus, there is one option that doesn’t work with getChild(). You can simply call removeChild() repeatedly until it returns null, indicating that there was no further such child. For example, this code fragment removes all the immediate note children of the Element named element:

while (element.removeChild("note") != null) ;

However, unlike the earlier example with removeChildren(), this is not recursive and will not find note elements deeper in the tree.

Getting and setting the text of an element

Sometimes what you want is the text of an element. For this purpose, JDOM provides these four methods:

public String getText();
public String getTextTrim();
public String getTextNormalize();
public Element setText(String text);

The getText() method returns the PCDATA content of the element. The getTextTrim() method returns pretty much the same content except that all leading and trailing whitespace has been removed. The getTextNormalize() method not only strips all leading and trailing whitespace; it also converts all runs of spaces to a single space. For example, consider this street element:

<street> 135  Airline  Highway </street>

For this element, getText() returns “ 135  Airline  Highway ” with the white space unchanged. However, getTextTrim() returns “135  Airline  Highway”, and getTextNormalize() returns “135 Airline Highway”. It’s an application level decision which one you want.

This is trickier than you might think at first glance. For instance, consider this street element:

<street>135<!-- The building doesn't actually have a number.
                It's next door to 133 -->Airline Highway</street>

getText() returns “135Airline Highway”. It ignores comments and processing instructions as if they weren’t there. For the most part that seems reasonable.

Now consider this street element:

<street>135 Airline Highway <apartment>2B</apartment></street>

getText() returns “135 Airline Highway ”. The content in the child apartment element is completely lost. This is not really a good thing. (I argued about this in the JDOM group, but I lost.) Before you can reliably use any of the getText()/getTextTrim()/ getTextNormalize() methods you need to be very sure that the element does not have any child elements. One way to do this is to test if the number of child elements is zero before invoking the text getter. For example,

if (element.getChildren().size() == 0) {
   String result = element.getText();
   // work with result …
}
else {
  // do something more complex …
}

An alternative is to write your own method that recursively descends through the element, accumulating all its text. I’ll demonstrate this in the section on the Text class shortly.

Do not use any of these getter methods unless you have first validated the document against a DTD or schema that explicitly requires the element only to contain #PCDATA. Do not assume that you “know” that this is true in your domain without individually testing each document. Invariably, sooner or later, you will encounter a document that purports to adhere to the implicit schema, and indeed is very close to it, but does not quite match what you were assuming. Explicit validation is necessary.

The setText() method is a little less fraught. You can set the text content of any element to whatever text you desire. For example, this code fragment sets the text of the street element to the string “3520 Airline Drive”:

street.setText("3520 Airline Drive");

This completely wipes out any existing content the element has: child elements, descendants, comments, processing instructions, other text, etc. If you just want to append the string to the existing text, use the addContent() method instead.

Getting child text

One common pattern in XML documents is an element that contains only other elements, all of which contain only PCDATA such as this channel element from Slashdot’s RSS file:

<channel>
  <title>Slashdot: News for nerds, stuff that matters</title>
  <link>http://slashdot.org/</link>
  <description>News for nerds, stuff that matters</description>
</channel>

Given such an element, JDOM provides six convenience methods for extracting the text, the trimmed text, and the normalized text from these child elements:

public String getChildText(String name);
public String getChildText(String name, Namespace namespace);
public String getChildTextTrim(String name);
public String getChildTextTrim(String name, Namespace namespace);
public String getChildTextNormalize(String name);
public String getChildTextNormalize(String name, Namespace namespace);

For example, assuming the Element object channel represents the above channel element, this code fragment retrieves the content of the title, link, and description elements:

String title = channel.getChildText("title");
String description = channel.getChildText("description");
String link = channel.getChildText("link");

There are two things I really don’t like about these methods. First, like the getText/getTextTrim/getTextNormalize() methods, they all fail unexpectedly and silently if any of the child elements unexpectedly contain child elements. For example, the above code fragment fails massively if Slashdot changes its format and begins distributing content like this instead:

<channel>
  <title>
    <trademark>Slashdot</trademark>
    <trademark>News for nerds, stuff that matters</trademark>
  </title>
  <link>http://slashdot.org/</link>
  <description>
    <trademark>News for nerds, stuff that matters</trademark>
  </description>
</channel>

Secondly, these methods fail unexpectedly and silently if the any of the child elements are repeated. For example, suppose instead the channel element has three link children like this:

<channel>
  <title>Slashdot: News for nerds, stuff that matters</title>
  <link>http://slashdot.org/</link>
  <link>http://www.slashdot.org/</link>
  <link>http://slashdot.com/</link>
  <description>News for nerds, stuff that matters</description>
</channel>

All three methods return the text from the first link element, and do not bother to inform the client program that there are more it may be interested in.

As with getText/getTextTrim/getTextNormalize(), do not use any of these methods without first validating the document against a DTD or schema that explicitly requires the child elements only to contain #PCDATA and to occur exactly once each in each parent element.

Filters

You can pass an org.jdom.filter.Filter object to the getContent() method to limit the content returned by the method. This interface, shown in Example 15.4, determines whether an object can be added to, removed from, or included in a particular list. For the purposes of navigation and search, only the matches() method really matters. It determines whether or not any particular object is included in the List returned by getContent(). The canAdd() and canRemove() methods test whether a particular object can be added to or removed from the list respectively. However, in the two default implementations of this class in ElementFilter and ContentFilter, both of these methods just call matches().

Example 15.4. The JDOM Filter interface

package org.jdom.filter;

public interface Filter {

  public boolean canAdd(Object o);
  public boolean canRemove(Object o);
  public boolean matches(Object o);

}

The org.jdom.filter package includes two implementations of this interface, ContentFilter (Example 15.5) and ElementFilter (Example 15.6). The ContentFilter class allows you to specify the visibility of different JDOM node types like ProcessingInstruction and Text. ElementFilter allows you to select elements with certain names or namespaces. Finally, you can write your own custom implementations that filter according to application-specific criteria.

Example 15.5. The ContentFilter class

package org.jdom.filter;

public class ContentFilter implements Filter {

  public static final int ELEMENT   = 1;
  public static final int CDATA     = 2;
  public static final int TEXT      = 4;
  public static final int COMMENT   = 8;
  public static final int PI        = 16;
  public static final int ENTITYREF = 32;
  public static final int DOCUMENT  = 64;

  protected int filterMask;

  public ContentFilter();
  public ContentFilter(boolean allVisible);
  public ContentFilter(int mask);

  public int  getFilterMask();
  public void setFilterMask(int mask);
  public void setDefaultMask();

  public void setDocumentContent();
  public void setElementContent();

  public void setElementVisible(boolean visible);
  public void setCDATAVisible(boolean visible)
  public void setTextVisible(boolean visible);
  public void setCommentVisible(boolean visible);
  public void setPIVisible(boolean visible);
  public void setEntityRefVisible(boolean visible);

  public boolean canAdd(Object o);
  public boolean canRemove(Object o);
  public boolean matches(Object o);

  public boolean equals(Object o);

}

For example, suppose your application only needs to concern itself with elements and text, but can completely skip all comments and processing instructions. You can simplify the code by using an appropriately configured ContentFilter. The most convenient approach is to construct a filter that filters out all nodes by passing false to the constructor, and then turn on only the types you want to let through like this:

// Filter out everything by default
Filter filter = new ContentFilter(false);
// Allow elements through the filter
filter.setElementVisible(true);
// Allow text nodes through the filter
filter.setTextVisible(true);

You’ll need to pass filter to getContent() every time you call it, like so:

  Filter filter; // set up in constructor

  public static void process(Element element) {

    List children = element.getContent(filter);
    Iterator iterator = children.iterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      if (o instanceof Element) {
        Element child = (Element) o;
        process(element);
      }
      else { // Due to filter, the only other possibility is Text
        Text text = (Text) o;
        handleText(text);
      }
    }

  }

You normally want to allow elements to pass the filter, even if you’re only looking at other things like Text. In JDOM recursing through the Element objects is the only way to search a complete tree. If you filter out the Elements, you won’t be able to go more than one level deep from where you start.

If you only want to select elements, you can use an ElementFilter instead. This can be set up to select all elements, elements with a certain name, elements in a certain namespace, or elements with a certain name in a certain namespace.

Example 15.6. The ElementFilter class

package org.jdom.filter;

public class ElementFilter implements Filter {

  protected String    name;
  protected Namespace namespace;

  public ElementFilter();
  public ElementFilter(String name);
  public ElementFilter(Namespace namespace);
  public ElementFilter(String name, Namespace namespace);

  public boolean canAdd(Object o);
  public boolean canRemove(Object o);
  public boolean matches(Object o);

  public boolean equals(Object o);

}

For example, this code fragment uses an ElementFilter to create a List named content that only contains XSLT elements:

Namespace xslt = Namespace.getNamespace(
                   "http://www.w3.org/1999/XSL/Transform");
Filter filter = new ElementFilter(xslt);
List content = element.getContent(filter);

Once again, however, this method proves to be less generally useful than the DOM equivalents because the getContent() method only returns children, not all descendants. For example, you couldn’t really use this to select the XSLT elements or the non-XSLT elements in a stylesheet because each type can appear as children of the other type.

Filters also work in the Document class, pretty much the same way they work in the Element class. For example, suppose you want to find all the processing instructions in the Document object doc outside the root element. This code fragment creates a List containing those:

// Filter out everything by default
Filter pisOnly = new ContentFilter(false);
// Allow processing instructions through the filter
pisOnly.setPIVisible(true);
// Get the content
List pis = doc.getContent(pisOnly);

If you want something a little more useful, like a filter that selects all xml-stylesheet processing instructions in the prolog only, then you need to write a custom implementation of Filter. Example 15.7 demonstrates.

Example 15.7. A filter for xml-stylesheet processing instructions in the prolog

import org.jdom.filter.Filter;
import org.jdom.*;
import java.util.List;

public class StylesheetFilter implements Filter {

  // This filter is read-only. Nothing can be added or removed.
  public boolean canAdd(Object o) {
    return false;
  }

  public boolean canRemove(Object o) {
    return false;
  }

  public boolean matches(Object o) {

    if (o instanceof ProcessingInstruction) {
      ProcessingInstruction pi = (ProcessingInstruction) o;
      if (pi.getTarget().equals("xml-stylesheet")) {
        // Test to see if we're outside the root element
        if (pi.getParent() == null) {
          Document doc = pi.getDocument();
          Element root = doc.getRootElement();
          List content = doc.getContent();
          if (content.indexOf(pi) < content.indexOf(root)) {
            // In prolog
            return true;
          }
        }
      }
    }
    return false;

  }

}

Adding and removing children

You can append any legal node to an Element using the 6-way overloaded addContent() methods:

public Element addContent(String s);
public Element addContent(Text text)
    throws IllegalAddException;
public Element addContent(Element element)
    throws IllegalAddException;
public Element addContent(ProcessingInstruction instruction)
    throws IllegalAddException;
public Element addContent(EntityRef ref)
    throws IllegalAddException;
public Element addContent(Comment comment)
    throws IllegalAddException;

These methods append their argument to Element’s child list. Except for addContent(String), they all throw an IllegalAddException if the argument already has a parent element. (The addContent(String) method is just a convenience that creates a new Text node behind the scenes. It does not actually add a String object to the content list.) All return the same Element object that invoked them which allows for convenient chaining.

These methods all add the new node to the end of the Element’s list. If you want to insert a node in a different position, you’ll have to retrieve the List object itself. For example, this code fragment creates the same channel element by inserting all the child nodes in reverse order at the beginning of the list using the add(int index, Object o) method:

Element channel     = new Element("channel");
Element link        = new Element("link");
Element description = new Element("description");
Element title       = new Element("title");
title.setText("Slashdot");
link.setText("http://slashdot.org/");
description.setText("News for nerds");

List content = channel.getContent();
content.add(0, description);
content.add(0, link);
content.add(0, title);

There are six removeContent() methods that remove a node from the list, wherever it resides:

public Element removeContent(Text text);
public Element removeContent(CDATA cdata);
public Element removeContent(Element element);
public Element removeContent(ProcessingInstruction instruction);
public Element removeContent(EntityRef ref);
public Element removeContent(Comment comment);

Of course, you can also retrieve the List from the Element with getContent() and remove elements by position using the list’s remove() and removeAll() methods. However, doing so is relatively rare. Normally you have or can easily get a reference to the specific node you want to remove. For example, this deletes the first link child element of the channel element:

channel.removeChild(channel.getChild("link"));

There is currently no method to remove all the content from an Element. Instead, just pass null to setContent(). That is,

element.setContent(null);

Parents and ancestors

So far we’ve mostly focused on moving down the tree using methods that return children and recursion. However, JDOM can also move up the tree as well. [2] As with the child-returning methods, you can only jump one level at a time. That is, you can only get the parent directly. To get other ancestor elements, you need to ask for the parent’s parent, the parent of the parent’s parent, and so forth, until eventually you find an element whose parent is null, which is of course the root of the tree.

Each Element object has zero or one parents. If the Element is the root element of the document (or at least the root of the tree in the event that the Element is not currently part of a Document), then this parent is null. Otherwise it is another Element object. JDOM does not consider the owner document to be the parent of the root element. These three methods enable you to determine whether or not an Element object represents a root element, and what its parent and owner document are:

public Document getDocument();
public boolean isRootElement();
public Element getParent();

Unlike DOM Elements, JDOM Elements are not irrevocably tied to their owner document. An Element may be in no document at all (in which case getDocument() returns null); and it may be moved from one document to another. However, JDOM Elements cannot have more than one parent at a time. Before you can move an element to a different Document or a different position in the same Document, you first have to detach it from its current parent by invoking the detach() method:

public Element detach();

After you’ve called detach(), you are free to add the Element to any other Element or Document. For example, Example 15.8 loads the XML document at http://www.slashdot.org/slashdot.rdf, detaches all the link elements from that document, and inserts them in a new linkset element, which it then outputs. Without the call to detach(), this would fail with an IllegalAddException.

Example 15.8. Moving elements between documents

import org.jdom.*;
import org.jdom.input.SAXBuilder;
import org.jdom.output.XMLOutputter;
import java.io.IOException;
import java.util.*;

public class Linkset {

  public static void main(String[] args) {

    String url = "http://www.slashdot.org/slashdot.rdf";

    try {
      SAXBuilder parser = new SAXBuilder();

      // Parse the document
      Document document = parser.build(url);
      Element oldRoot = document.getRootElement();
      Element newRoot = new Element("linkset");
      List content = oldRoot.getChildren();
      Iterator iterator = content.iterator();
      while (iterator.hasNext()) {
        Object next = iterator.next();
        Element element = (Element) next;
        Element link = element.getChild("link",
         Namespace.getNamespace(
         "http://my.netscape.com/rdf/simple/0.9/"));
        link.detach();
        newRoot.addContent(link);
      }

      XMLOutputter outputter = new XMLOutputter("  ", true);
      outputter.output(newRoot, System.out);
    }
    catch (JDOMException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not read " + url
      );
    }

  } // end main

}

As usual, this only affects the JDOM Document object in memory. It has no effect on the original document read from the remote URL.

Another natural limitation is that an element cannot be its own parent or ancestor, directly or indirectly. Trying to add an element where it would violate this restriction throws an IllegalAddException. You can test whether one element is an ancestor of another using the isAncestor() method:

public boolean isAncestor(Element element);

Attributes

The Element class has thirteen methods that read and write the values of the various attributes of the element. Except for certain unusual cases (mostly involving attribute types) these thirteen methods are all that’s needed to handle attributes. You rarely need to concern yourself with the Attribute class directly.

public Attribute getAttribute(String name);
public Attribute getAttribute(String name, Namespace namespace);
public String getAttributeValue(String name);
public String getAttributeValue(String name, Namespace namespace);
public String getAttributeValue(String name, String default);
public String getAttributeValue(String name, Namespace namespace, String default);
public Element setAttributes(List attributes)
    throws IllegalAddException;
public Element setAttribute(String name, String value)
    throws IllegalNameException, IllegalDataException;
public Element setAttribute(String name, String value, Namespace namespace)
    throws IllegalNameException, IllegalDataException;
public Element setAttribute(Attribute attribute)
    throws IllegalAddException;
public boolean removeAttribute(String name, String value);
public boolean removeAttribute(String name, Namespace namespace);
public boolean removeAttribute(Attribute attribute);

These methods all follow the same basic rules. If an attribute is in a namespace, specify the local name and namespace to access it. If the attribute is not in a namespace, then only use the name. The setters must also specify the value to set the attribute to. The getters may optionally specify a default value used if the attribute is not found. Alternately, you can use an Attribute object to replace all of these. Most of the time, however, strings are more convenient.

The getAttributeValue() methods all return the String value of the attribute. If the attribute was read by a parser, the value will be normalized according to its type. However, attributes added in-memory with setAttribute() and its ilk will not be normalized. The setter methods all return the Element object itself so they can be used in a chain. The remove methods all return a boolean, true if the attribute was removed, false if it wasn’t.

As with most other constructs, JDOM checks all the attributes you set for well-formedness and throws an exception if anything looks amiss. In particular,

  • The local name must be a non-colonized name.

  • The value can’t contain any illegal characters like null or the byte order mark.

  • The attribute cannot be a namespace declaration such as xmlns or xmlns:prefix. (JDOM stores these separately.)

For example, let’s suppose you want to process a RDDL document to find resources related to a particular namespace URI. Each of these is enclosed in a rddl:resource element like this one from the RDDL specification itself:

<rddl:resource xlink:type="simple"
        xlink:title="RDDL Natures"
        xlink:role="http://www.rddl.org/"
        xlink:arcrole="http://www.rddl.org/purposes#directory"
        xlink:href="http://www.rddl.org/natures"
>
<div class="resource">
<p>It is anticipated that many related-resource natures will be
   well known. A list of well-known natures may be found in the
   RDDL directory <a href=
   "http://www.rddl.org/natures">http://www.rddl.org/natures</a>.
</p>
</div>
</rddl:resource>

All the information required to locate the resources is included in the attributes of the rddl:resource elements. The rest of the content in the document is relevant only to a browser showing the document to a human reader. Most software will want to read these rddl:resource elements and ignore the rest of the document. Example 15.9 is such a program. It searches a document for related resources and outputs an HTML table containing their information. The xlink:href attribute becomes an HTML hyperlink. The other URLs in the xlink:role and xlink:arcrole attributes are purely descriptive (like namespace URLs) and not intended to be resolved, so they’re merely output as plain text.

Example 15.9. Searching for RDDL resources

import org.jdom.*;
import org.jdom.input.SAXBuilder;
import org.jdom.output.XMLOutputter;
import java.util.*;
import java.io.IOException;

public class RDDLLister {

  public final static Namespace XLINK_NAMESPACE =
   Namespace.getNamespace("xl", "http://www.w3.org/1999/xlink");
  public final static String RDDL_NAMESPACE
   = "http://www.rddl.org/";

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java RDDLLister url");
      return;
    }

    SAXBuilder builder = new SAXBuilder();

    try {
      // Prepare the output document
      Element html = new Element("html");
      Element body = new Element("body");
      Element table = new Element("table");
      html.addContent(body);
      body.addContent(table);
      Document output = new Document(html);

      // Read the entire document into memory
      Document doc = builder.build(args[0]);
      Element root = doc.getRootElement();
      processElement(root, table);

      // Serialize the output document
      XMLOutputter outputter = new XMLOutputter("  ", true);
      outputter.output(output, System.out);

    }
    catch (JDOMException e) {
      System.err.println(e);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  } // end main

  public static void processElement(Element input, Element output) {

    if (input.getName().equals("resource")
     && input.getNamespaceURI().equals(RDDL_NAMESPACE)) {

       String href    = input.getAttributeValue("href", XLINK_NAMESPACE);
       String title   = input.getAttributeValue("title", XLINK_NAMESPACE);
       String role    = input.getAttributeValue("role", XLINK_NAMESPACE);
       String arcrole = input.getAttributeValue("arcrole", XLINK_NAMESPACE);

       // Wrap this up in a table row
       Element tr = new Element("tr");

       Element titleCell = new Element("td");
       titleCell.setText(title);
       tr.addContent(titleCell);

       Element hrefCell = new Element("td");
       Element a = new Element("a");
       a.setAttribute("href", href);
       a.setText(href);
       hrefCell.addContent(a);
       tr.addContent(hrefCell);

       Element roleCell = new Element("td");
       roleCell.setText(role);
       tr.addContent(roleCell);

       Element arcroleCell = new Element("td");
       arcroleCell.setText(arcrole);
       tr.addContent(arcroleCell);

       output.addContent(tr);

    }

    // Recurse
    List content = input.getContent();
    Iterator iterator = content.iterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      if (o instanceof Element) {
        processElement((Element) o, output);
      }
    } // end while

  }

}

The main() method builds the general outline of a well-formed HTML document, and then parses the input RDDL document in the usual fashion. It retrieves the root element with getRootElement() and then passes this root element and the table element to the processElement() method.

First processElement() checks to see if the element is a rddl:resource element. If it is, then processElement() extracts the four XLink attributes using getAttributeValue(). Each of these is then inserted in a td element which is appended to a tr element which is added to the table element. The setAttribute() method attaches an href attribute to the a element that defines the HTML link. Finally, the processElement() method is invoked on all child elements of the current elements to find any rddl:resource elements that are deeper down the tree.

Here’s the beginning of output from when I ran this program against the RDDL specification itself:

D:\books\XMLJAVA>java RDDLLister http://www.rddl.org
<?xml version="1.0" encoding="UTF-8"?>
<html>
  <body>
    <table>
      <tr>
        <td>RDDL Natures</td>
        <td>
          <a href="http://www.rddl.org/natures">
           http://www.rddl.org/natures</a>
        </td>
        <td>http://www.rddl.org/</td>
        <td>http://www.rddl.org/purposes#directory</td>
      </tr>
      <tr>
        <td>RDDL Purposes</td>
        <td>
          <a href="http://www.rddl.org/purposes">
           http://www.rddl.org/purposes</a>
        </td>
        <td>http://www.rddl.org/</td>
        <td>http://www.rddl.org/purposes#directory</td>
      </tr>
…

[1] The name is a little misleading. An earlier beta called these methods getChildElements() and removeChildElements(), much better names in my opinion.

[2] Sideways movement, e.g. getting the previous or next sibling, is noticeably lacking. For this, you normally use List and Iterator.

Prev | Up | Next
Chapter 15. The JDOM Model | Home | The Attribute Class
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 12, 2002


The Attribute Class
Prev Chapter 15. The JDOM Model Next

The Attribute Class

The Attribute class, shown in Example 15.10, represents an attribute of an element, other than one that declares a namespace. Each attribute has these five basic properties.

Local name

A String accessible through the setName() and getName() methods

Namespace

A Namespace object which encapsulates both the namespace URI and prefix. This is accessible through the setNamespace() and getNamespace() methods. For all unprefixed attributes this is Namespace.NO_NAMESPACE. The prefix, URI, and fully qualified name are separately readable through the getNamespaceURI(), getNamespacePrefix(), and getQualifiedName() convenience methods.

Value

A String containing the attribute’s normalized value. This is accessible through the getValue() and setValue() methods. The unnormalized value is not available. There are also convenience methods that read the attribute value as a double, float, long, int, or boolean.

Parent

The Element that possesses this Attribute. This is accessible through the getParent() and setParent() methods. An Attribute cannot have more than one parent. Before attaching a new parent Element, you must first invoke detach() to remove the Attribute from its current parent.

Type

The Attribute’s type, as specified in the DTD. This is one of the ten named constants Attribute.CDATA_ATTRIBUTE, Attribute.ID_ATTRIBUTE, Attribute.IDREFS_ATTRIBUTE, Attribute.IDREFS_ATTRIBUTE, Attribute.ENUMERATED_ATTRIBUTE, and so forth. If the DTD does not specify the attribute’s type, then the type is set to Attribute.UNDECLARED_ATTRIBUTE.[3] The getAttributeType() and setAttributeType() methods access this property.

In addition, you can get the Document to which the Attribute belongs with the getDocument() method. However, this is not really independent of the Element to which the attribute is attached.

Example 15.10. The JDOM Attribute class

package org.jdom;

public class Attribute implements Serializable, Cloneable {

  public final static int UNDECLARED_ATTRIBUTE = 0;
  public final static int CDATA_ATTRIBUTE      = 1;
  public final static int ID_ATTRIBUTE         = 2;
  public final static int IDREF_ATTRIBUTE      = 3;
  public final static int IDREFS_ATTRIBUTE     = 4;
  public final static int ENTITY_ATTRIBUTE     = 5;
  public final static int ENTITIES_ATTRIBUTE   = 6;
  public final static int NMTOKEN_ATTRIBUTE    = 7;
  public final static int NMTOKENS_ATTRIBUTE   = 8;
  public final static int NOTATION_ATTRIBUTE   = 9;
  public final static int ENUMERATED_ATTRIBUTE = 10;

  protected           String    name;
  protected transient Namespace namespace;
  protected           String    value;
  protected           int       type;
  protected           Object    parent;

  protected Attribute();

  public Attribute(String name, String value,
   Namespace namespace);
  public Attribute(String name, String value, int type,
   Namespace namespace);
  public Attribute(String name, String value);
  public Attribute(String name, String value, int type);

  public Document     getDocument();
  public Element      getParent();
  protected Attribute setParent(Element parent);
  public Attribute    detach();
  public String       getName();
  public Attribute    setName(String name);
  public String       getQualifiedName();
  public String       getNamespacePrefix();
  public String       getNamespaceURI();
  public Namespace    getNamespace();
  public Attribute    setNamespace(Namespace namespace);
  public String       getValue();
  public Attribute    setValue(String value);
  public int          getAttributeType();
  public Attribute    setAttributeType(int type);

  public String        toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public Object        clone();

  public int     getIntValue() throws DataConversionException;
  public long    getLongValue() throws DataConversionException;
  public float   getFloatValue() throws DataConversionException;
  public double  getDoubleValue() throws DataConversionException;
  public boolean getBooleanValue() throws DataConversionException;

}

Tip

One of the key things to remember when working with attributes, whether in JDOM or any other XML technology, is that unprefixed attributes are never in any namespace. In particular,

  1. Attributes are never in the default namespace.

  2. An attribute is not in the same namespace as its parent element (except in the unusual case where it happens to have the same prefix as its parent element).

90% or more of the time, you’re just going to use the the set/get/removeAttribute() methods in the Element class rather than using the Attribute class. The only major case where you really need to use the Attribute class directly is when the attribute type matters; for instance when you want to treat an ID-type attribute differently than a CDATA attribute or a NOTATIONS attribute differently than a NMTOKENS attribute.

For example, consider attribute value normalization. When a parser reports an attribute value to the client application it adjusts the white space according to the attribute type. Attributes of type CDATA and undeclared attributes preserve all white space. However, white space is trimmed from the edges of all other attribute types, and all runs of white space are compressed to a single space. For instance, consider this fact start-tag:

<fact source=" f21 f32   f33
 f122 f87 f893  ">

If the document’s DTD declares that the source attribute has type IDREFS, then the parser will report its value as f21 f32 f33 f122 f87 f893. On the other hand, if the DTD declares that it has type CDATA or does not assign it a type at all, then the parser will report its value with all the spaces and line breaks intact.

JDOM will accept whatever value the parser initially reports when the document is constructed. However, if attributes are added or their values modified later, then attributes will no longer be in normalized form. To fix this, we can write a method that searches a document for attributes whose type is something other than CDATA and normalizes their space. As usual the method is recursive:

  public void normalizeAttributes(Element element) {

    List attributes = element.getAttributes();
    Iterator iterator = attributes.iterator();
    while (iterator.hasNext()) {
      Attribute attribute = (Attribute) iterator.next();
      int type = attribute.getAttributeType();
      if (type != Attribute.CDATA_ATTRIBUTE
       && type != Attribute.UNDECLARED_ATTRIBUTE) {
         String oldValue = attribute.getValue();
         String newValue = Text.normalizeString(oldValue);
         attribute.setValue(newValue);
      }
    }

    List content = element.getContent();
    Iterator children = content.iterator();
    while (children.hasNext()) {
      Object o = children.next();
      if (o instanceof Element) {
        Element child = (Element) o;
        normalizeAttributes(child);
      }
    }

  }

The actual normalization is performed by the static Text.normalizeString() method from the Text class.

[3] In practice, SAX gets this wrong. It does not distinguish between CDATA type attributes and undeclared attributes. Thus when a SAX parser builds a JDOM Document, no attributes will have type Attribute.UNDECLARED_ATTRIBUTE.

Prev | Up | Next
The Element Class | Home | The Text Class
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified April 30, 2002


The Text Class
Prev Chapter 15. The JDOM Model Next

The Text Class

JDOM uses the Text class internally to represent text nodes. In normal usage you don’t deal with this class directly. You just use strings. The one time you may encounter it is when you use getContent() to retrieve all the children of the node, and you’re iterating through the list returned. In this case, you will see Text objects.

Each Text object has a parent Element (which may be null) and a String value that holds the content of the node. This value may contain characters like < and &. If so, they will be escaped when the node is serialized. However they do not need to be escaped before inserting them into a Text object.

The Text class, summarized in Example 15.11, has methods to get, set, and detach the parent Element, to get and set the text content as a String, to append more text to the node, and to get the content of the node after trimming or normalizing white space. And of course, it has the other usual Java methods such as equals(), hashCode(), and clone() that all JDOM objects possess.

Example 15.11. The JDOM Text class

package org.jdom;

public class Text implements Serializable, Cloneable {

  protected String value;
  protected Object parent;

  protected Text();
  public    Text(String s);

  public String getText();
  public String getTextTrim();
  public String getTextNormalize();

  public static String normalizeString(String s);

  public Text     setText(String s);
  public void     append(String s);
  public void     append(Text text);
  public Element  getParent();
  public Document getDocument();
  protected Text  setParent(Element parent);
  public Text     detach();

  public       String  toString();
  public final int     hashCode();
  public final boolean equals(Object ob);
  public       Object  clone();

}

JDOM does not guarantee that each run of text is represented by a single text node. Text objects can be adjacent to each other. This can make it a little tricky to retrieve the complete content of an element. For example, consider this element:

  <vendor>
    Gus's  Crawfish
  </vendor>

Just from looking at the XML, there’s no way to say whether the Element object representing the vendor element contains one Text object or two. Indeed, in extreme cases, it may contain three, four, or even more. If this element was read by SAXBuilder, then JDOM does use a single Text object. However, if it was created or modified in memory by a program, then all bets are off.

In fact, you’d need to concern yourself with this even if JDOM did not allow adjacent text nodes. For example, consider this element:

  <vendor>
    Gus's <!-- This is my brother-in-law. My wife asked me to
         throw him some business. --> Crawfish
  </vendor>

The text content of the vendor element is the same as before. However, now there’s no way for JDOM to represent it as a single Text object.

You must also consider the case where an element contains child elements such as this one:

  <vendor>
    Gus's <seafood>Crawfish</seafood>
  </vendor>

To accumulate the complete text of an element you need to iterate through its children, while recursively processing any element children. This getFullText() method demonstrates:

  public static String getFullText(Element element) {

    StringBuffer result = new StringBuffer();
    List content = element.getContent();
    Iterator iterator = content.iterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      if (o instanceof Text) {
        Text t = (Text) o;
        result.append(t.getText());
      }
      else if (o instanceof Element) {
        Element child = (Element) o;
        result.append(getFullValue(child));
      }
    }

    return result.toString();

  }

Chapter 11 demonstrated a program that encoded all the text of a document, but not its markup, in ROT-13 using DOM. Let’s repeat that example here, but now with JDOM instead. You can compare it to Example 11.8 to get a good feeling for the differences between DOM and JDOM. The DOM version is significantly more complex, especially when it comes to building the document and then serializing it.

Example 15.12. JDOM based ROT13 encoder for XML documents

import org.jdom.*;
import org.jdom.output.XMLOutputter;
import org.jdom.input.SAXBuilder;
import java.io.IOException;
import java.util.*;

public class ROT13XML {

  // note use of recursion
  public static void encode(Element element) {

    List content = element.getContent();
    Iterator iterator = content.iterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      if (o instanceof Text) {
        Text t = (Text) o;
        String cipherText = rot13(t.getText());
        t.setText(cipherText);
      }
      else if (o instanceof Element) {
        Element child = (Element) o;
        encode(child);
      }
    }

  }

  public static String rot13(String s) {

    StringBuffer out = new StringBuffer(s.length());
    for (int i = 0; i < s.length(); i++) {
      int c = s.charAt(i);
      if (c >= 'A' && c <= 'M') out.append((char) (c+13));
      else if (c >= 'N' && c <= 'Z') out.append((char) (c-13));
      else if (c >= 'a' && c <= 'm') out.append((char) (c+13));
      else if (c >= 'n' && c <= 'z') out.append((char) (c-13));
      else out.append((char) c);
    }
    return out.toString();

  }

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java ROT13XML URL");
      return;
    }

    String url = args[0];

    try {
      SAXBuilder parser = new SAXBuilder();

      // Read the document
      Document document = parser.build(url);

      // Modify the document
      ROT13XML.encode(document.getRootElement());

      // Write it out again
      XMLOutputter outputter = new XMLOutputter();
      outputter.output(document, System.out);
    }
    catch (JDOMException e) {
      System.out.println(url + " is not well-formed.");
    }
    catch (IOException e) {
      System.out.println(
       "Due to an IOException, the parser could not encode " + url
      );
    }

  } // end main

}

Here’s a joke encoded by this program. You’ll have to run the program if you want to find out what it says. :-)

D:\books\XMLJAVA>java ROT13XML joke.xml
<?xml version="1.0" encoding="UTF-8"?>
<joke>
  Gur qrsvavgvba bs n yvoregnevna vf n pbafreingvir
  haqre vaqvpgzrag.
</joke>
Prev | Up | Next
The Attribute Class | Home | The CDATA Class
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 18, 2002


The CDATA Class
Prev Chapter 15. The JDOM Model Next

The CDATA Class

The CDATA class shown in Example 15.13 is a subclass of Text with almost no functionality of its own. The only difference between CDATA and Text is that when an XMLOutputter serializes a CDATA object, it places its contents in a CDATA section rather than escaping reserved characters such as < with character or entity references.

Example 15.13. The JDOM CDATA class

package org.jdom;

public class CDATA extends Text {

  protected CDATA() { }
  public    CDATA(String s) throws IllegalDataException;

  public Text setText(String s) throws IllegalDataException;
  public void append(String s) throws IllegalDataException;

  public String toString();

}

In my opinion, you really shouldn’t use this class at all. The builder may (or may not) create CDATA objects when it parses a document that contains CDATA sections. However, you should not create them yourself. CDATA sections are purely a convenience for human authors. They are not part of the document’s infoset. They should not be exposed as a separate item in the logical model of a document, and indeed not all parsers and APIs will report them to the client program. Even APIs like JDOM and DOM that support them do not necessarily guarantee that they’ll be used where possible.

Chapter 11 already warned you not to not use CDATA sections as a sort of pseudo-element to hide HTML in your XML documents. That warning bears repeating now. CDATA sections let you add non-well-formed text to a document, but their contents are just text like any other text. They are not a special kind of element, and a parser more likely than not won’t distinguish between the contents of the CDATA section and the surrounding text. If you have a legitimate reason for doing this, you still need to enclose the CDATA section in an actual element to provide structure programs can detect. For example, an HTML tutorial might enclose HTML code fragments or complete documents in example elements like this:

<example>
<![CDATA[<html>
<body>
  <h1>My First Web Page</h1>

  HTML is cool!<P>
  <hr>
  &copy; 2002 John Smith
  </body>
</html>]]>
</example>

This is much more flexible and much more robust than relying on CDATA sections to distinguish the examples from the main body text.

Prev | Up | Next
The Text Class | Home | The ProcessingInstruction Class
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 04, 2002


The ProcessingInstruction Class
Prev Chapter 15. The JDOM Model Next

The ProcessingInstruction Class

The ProcessingInstruction class represents an XML processing instruction such as <?xml-stylesheet href="book.xsl" type="application/xml"?> or <?php mysql_connect("database.unc.edu", "debra", "secret"); ?>. Each processing instruction has a target string, which must be a legal XML name, a value string, which may not contain the end-delimiter ?>, and a parent element (which may be null if the instruction is a child of a Document instead of an Element).

Note

At the time of this writing, JDOM only checks processing instruction targets for well-formedness. It does not check processing instruction data. However, checking data for the presence of illegal characters is on the TODO list.

Because so many (though not all) processing instructions have the pseudo-attribute format exemplified by <?xml-stylesheet href="book.xsl" type="application/xml"?>, this class also offers a second view of the instruction’s data, as a map of name-value pairs. Internally, ProcessingInstruction always stores the string form but only stores the map form if it can successfully parse the data as pseudo-attributes. If a program uses the map-based methods like getValue() and removeValue() but the actual format of the data is not pseudo-attributes, then they return null (getters). Using a setter method replaces the existing data with a pseudo-attribute.

Example 15.14. The JDOM ProcessingInstruction class

package org.jdom;

public class ProcessingInstruction
 implements Serializable, Cloneable {

  protected String target;
  protected String rawData;
  protected Map    mapData;
  protected Object parent;

  protected ProcessingInstruction();
  public    ProcessingInstruction(String target, Map data);
  public    ProcessingInstruction(String target, String data);

  public Element               getParent();
  public ProcessingInstruction detach();
  public Document              getDocument();

  public String getTarget();
  public String getData();
  public List   getNames();
  public String getValue(String name);

  public ProcessingInstruction setTarget(String target)
   throws IllegalTargetException;
  public ProcessingInstruction setData(String data);
  public ProcessingInstruction setData(Map data);
  public ProcessingInstruction setValue(String name,
   String value);

  public boolean removeValue(String name);

  public String        toString();
  public final boolean equals(Object ob);
  public final int     hashCode();
  public Object        clone();

}

For example, robots and spiders that process XML documents are supposed to recognize processing instructions in the form <?robots index="yes | no" follow="yes | no"?> to decide whether or not to index and/or follow the links in an XML document. Such a robot could use this method to decide whether to index a document it had parsed:

  public static boolean canIndex(Document doc) {

    List content = doc.getContent();
    Iterator iterator = content.iterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      if (o instanceof ProcessingInstruction) {
        ProcessingInstruction pi = (ProcessingInstruction) o;
        if (pi.getTarget().equals("robots")) {
          if ("no".equals(pi.getValue("index"))) return false;
        }
      }
      else if (o instanceof Element) {
        // This is the root element. The prolog is done.
        break;
      }
    }

    return true;

  }

This method looks for the first robots processing instruction in the document’s prolog. When it finds it, it requests the value of the index pseudo-attribute and returns true if it has the value yes or the value null and false if it has the value no. If no robots processing instruction is present, or it has the wrong format, the default is to assume that indexing is permitted. A canSpider() method is almost identical except that you’d ask for the value of the follow pseudo-attribute instead.

Prev | Up | Next
The CDATA Class | Home | The Comment Class
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 03, 2002


The Comment Class
Prev Chapter 15. The JDOM Model Next

The Comment Class

The org.jdom.Comment class represents an XML comment such as <-- Remember to verify this -->. As you can see from Example 15.15, Comment is really a very simple class that contains some string data, the usual getParent() and getDocument() methods, and the customary Java utility methods like equals() and toString().

Example 15.15. The JDOM Comment class

package org.jdom;

public class Comment implements Serializable, Cloneable {

  protected String text;
  protected Object parent;

  protected Comment();
  public Comment(String text);

  public Element    getParent();
  protected Comment setParent(Element parent);
  public Comment    detach();
  public Document   getDocument();
  protected Comment setDocument(Document document);
  public String     getText();
  public Comment    setText(String text);

  public       String  toString();
  public final boolean equals(Object ob);
  public final int     hashCode();
  public       Object  clone();

}

For example, this code fragment adds the comment <--An example from Chapter 15 of Processing XML with Java--> at the top of the Document object doc:

Comment comment = new Comment(
 "An example from Chapter 15 of Processing XML with Java");
List content = doc.getContent();
content.add(0, comment);

As with the other JDOM node classes, JDOM does attempt to verify that any data you supply for a Comment is well-formed. There are really only two constraints that matter for comments:

  • The comment cannot contain any illegal characters such as a null or a vertical tab.

  • The comment cannot contain the double hyphen -- that signals the end of the comment.

JDOM checks both of them.

There’s not a lot you can do with comments, nor is there a lot you should do with them. Comments are intended purely as a convenience for human authors. Programs really shouldn’t consider them or attempt to parse their contents. The only reason they’re in the API at all is to support round-tripping between the document that’s read and the document that’s written. Thus the examples are going to be fairly simple.

Earlier in the book, you saw SAX and DOM programs that printed the comments in an XML document on System.out. Now in Example 15.16 you can see the JDOM equivalent. The pattern is very much the same as in the DOM version, recursively descending the tree looking for objects of type Comment. However, the detailed classes are different.

Example 15.16. Printing comments

import org.jdom.*;
import org.jdom.input.SAXBuilder;
import java.util.*;
import java.io.IOException;

public class JDOMCommentReader {

  public static void main(String[] args) {

    if (args.length <= 0) {
      System.out.println("Usage: java JDOMCommentReader url");
      return;
    }
    SAXBuilder builder = new SAXBuilder();

    try {
      // Read the entire document into memory
      Document doc = builder.build(args[0]);
      List content = doc.getContent();
      Iterator iterator = content.iterator();
      while (iterator.hasNext()) {
        Object o = iterator.next();
        if (o instanceof Comment) {
          Comment c = (Comment) o;
          System.out.println(c.getText());
          System.out.println();
        }
        else if (o instanceof Element) {
          processElement((Element) o);
        }
      }
    }
    catch (JDOMException e) {
      System.err.println(e);
    }
    catch (IOException e) {
      System.err.println(e);
    }

  } // end main

  // note use of recursion
  public static void processElement(Element element) {

    List content = element.getContent();
    Iterator iterator = content.iterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      if (o instanceof Comment) {
        Comment c = (Comment) o;
        System.out.println(c.getText());
        System.out.println();
      }
      else if (o instanceof Element) {
        processElement((Element) o);
      }
    } // end while

  }

}

Here’s the result of running this program on the XLink specification:

D:\books\XMLJAVA>java JDOMCommentReader
 http://www.w3.org/TR/2001/REC-xlink-20010627/Overview.xml

Last edited: 19 December 2000 by elm

TO DO:
- Point to the linking/style Note if it gets published in time

http://www.w3.org/TR/2000/CR-xlink-20000703/
http://www.w3.org/TR/WD-xlink-20000221
http://www.w3.org/TR/WD-xlink-20000119
http://www.w3.org/TR/WD-xlink-19991220
http://www.w3.org/1999/07/WD-xlink-19990726
http://www.w3.org/TR/1998/WD-xlink-19980303
http://www.w3.org/TR/WD-xml-link-970731
Prev | Up | Next
The ProcessingInstruction Class | Home | Namespaces
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 06, 2002


Namespaces
Prev Chapter 15. The JDOM Model Next

Namespaces

JDOM’s Namespace class, shown in Example 15.17, represents a namespace attached to an element or an attribute. Each namespace has a URI. If the namespace is not a default namespace, then it also has a prefix. Otherwise the prefix is the empty string.

Example 15.17. The JDOM Namespace class

package org.jdom;

public final class Namespace {

  // Common namespaces
  public static final Namespace NO_NAMESPACE;
  public static final Namespace XML_NAMESPACE;

  // Factory methods
  public static Namespace getNamespace(String prefix, String uri);
  public static Namespace getNamespace(String uri);

  // Getter methods
  public String getPrefix();
  public String getURI();

  // Utility methods
  public boolean equals(Object o);
  public String toString();
  public int hashCode();

}

Because repeating long strings like “http://www.w3.org/2002/01/P3Pv1” on each element can eat up memory very quickly, and because a typical document contains many elements in the same namespace, this class uses the flyweight design pattern. This means the constructors are private, and you’ll need to use the factory methods to create Namespace objects. Alternately, the Element and Attribute classes allocate or reuse the necessary Namespace objects automatically when you pass string forms of the namespace URIs to their constructors or setNamespace() methods.

For an example, I try to use well-formed HTML on most of my sites. However, I generally don’t attach the XHTML namespace (http://www.w3.org/1999/xhtml) where I should. The following method forces all unqualified elements into the XHTML namespace:

  public static void xhtmlQualify(Element element) {

    Namespace xhtml
     = Namespace.getNamespace("http://www.w3.org/1999/xhtml");

    if (element.getNamespace() == Namespace.NO_NAMESPACE) {
      element.setNamespace(xhtml);
    }

    List childElements = element.getChildren();
    Iterator iterator = childElements.iterator();
    while (iterator.hasNext()) {
      Element child = (Element) iterator.next();
      xhtmlQualify(child);
    }

  }

This changes the Document object in place (though not the original file on disk or the network). It does not create any new node objects. You could use this as a filter before passing the Document object to a different method that expected pure XHTML. (In practice, you’d probably also need to change all the element names to lower case and add a DocType.)

You normally don’t need to worry about exactly where the xmlns and xmlns:prefix attributes that declare namespaces are placed. (Indeed JDOM won’t let you add attributes with these names since it stores them separately from the other attributes.) When an outputter converts a JDOM Document to a DOM Document, a SAX event sequence, or a stream of bytes, it will figure out where it needs to put namespace declarations to make everything come out right. However, some XML applications, including SOAP, XSLT, and the W3C XML Schema Language, also use namespace prefixes in attribute values and even element content. These prefixes are not necessarily used on any element or attribute names in the document. However, the prefixes still need to be declared. For example, the simple XSLT stylesheet in Example 15.18 needs to declare the prefix svg even though it’s only used in the value of the match attribute.

Example 15.18. An XML document that uses namespace prefixes in attribute values

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:svg="http://www.w3.org/2000/svg">

  <xsl:template match="svg:rect">
    <rectangle><xsl:apply-templates/></rectangle>
  </xsl:template>

</xsl:stylesheet>

You can add these extra namespace bindings through the addNamespaceDeclaration() method in the Element class. If necessary, you can remove one with the removeNamespaceDeclaration() method:

public Element addNamespaceDeclaration(Namespace namespace);
public Element removeNamespaceDeclaration(Namespace namespace);

For example, this code fragment creates the above xsl:stylesheet element and adds the SVG namespace declaration to it:

Element stylesheet = new Element(
    "stylesheet", "xsl", "http://www.w3.org/1999/XSL/Transform");
Namespace svg = Namespace.getNamespace("svg",
                                   "http://www.w3.org/2000/svg");
stylesheet.addNamespaceDeclaration(svg);

If you encounter a namespace prefix in character data, and you need to know what prefix it maps to, you have to check the parent element’s namespace, all its attributes’ namespaces, and all its additional namespaces. If that doesn’t give you an answer, repeat the process for the next nearest ancestor, and continue until you either find the answer or run out of ancestors. This can be a little involved, but fortunately the Element class provides a simple method that tells you what URI any given prefix maps to within its scope:

public Namespace getNamespace(String prefix);

Since prefixes can be remapped to different URIs in descendant elements, always make sure to check the namespace in scope from the Attribute or Text object’s immediate parent.

Prev | Up | Next
The Comment Class | Home | The DocType class
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 30, 2002


The DocType class
Prev Chapter 15. The JDOM Model Next

The DocType class

The org.jdom.DocType class summarized in Example 15.19 represents a document type declaration. Note that this points to and/or contains the document type definition (DTD). However, it is not the same thing. JDOM does not have any representation of the DTD.

Example 15.19. The JDOM DocType class

package org.jdom;

public class DocType implements Serializable, Cloneable {

  protected String   elementName;
  protected String   publicID;
  protected String   systemID;
  protected Document document;
  protected String   internalSubset;

  protected DocType();
  public DocType(String elementName, String publicID,
   String systemID);
  public DocType(String elementName, String systemID);
  public DocType(String elementName);

  public String   getElementName();
  public DocType  setElementName(String elementName);
  public String   getPublicID();
  public DocType  setPublicID(String publicID);
  public String   getSystemID();
  public DocType  setSystemID(String systemID);
  public Document getDocument();
  public void     setInternalSubset(String newData);

  public String getInternalSubset();

  public       String  toString();
  public final boolean equals(Object o);
  public final int     hashCode();
  public       Object  clone();

}

Each DocType object has four String properties, of which the last three may be null:

  • root element name

  • internal DTD subset

  • system ID

  • public ID

For example, consider this document type declaration:

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
                         "docbook/docbookx.dtd">

It has the root element name chapter, the public ID -//OASIS//DTD DocBook XML V4.1.2//EN, and the system ID docbook/docbookx.dtd. However, its internal DTD subset is null. This code fragment constructs a DocType object representing this document type declaration and uses it to construct a new Document object:

DocType doctype = new DocType("chapter",
 "-//OASIS//DTD DocBook XML V4.1.2//EN", "docbook/docbookx.dtd");
Element chapter = new Element("chapter");
Document doc = new Document(chapter, doctype);

However, JDOM does not require validity, only well-formedness. This means that the root element may in fact be different than what the document type declaration specifies. For example, this is perfectly legal:

DocType doctype = new DocType("chapter",
 "-//OASIS//DTD DocBook XML V4.1.2//EN", "docbook/docbookx.dtd");
Element book = new Element("book");
Document doc = new Document(book, doctype);

This document type declaration has a root element name and an internal DTD subset, but no public ID or system ID:

<!DOCTYPE Fibonacci_Numbers [
  <!ELEMENT Fibonacci_Numbers (fibonacci*)>
  <!ELEMENT fibonacci (#PCDATA)>
  <!ATTLIST fibonacci index CDATA #IMPLIED>
]>

To set this up, you need to store the internal subset in a String and pass that to the setInternalSubset() method after the DocType object has been constructed like so:

DocType doctype = new DocType("Fibonacci_Numbers");
String dtd = "<!ELEMENT Fibonacci_Numbers (fibonacci*)>\n";
dtd += "<!ELEMENT fibonacci (#PCDATA)>\n";
dtd += "<!ATTLIST fibonacci index CDATA #IMPLIED>\n";
doctype.setInternalSubset(dtd);
Element root = new Element("Fibonacci_Numbers");
Document doc = new Document(root, doctype);

Unlike most node classes, JDOM doesn’t fully check the data used in a DocType object for well-formedness. It does test that the root element name is a legal XML name, and it checks that the public and system IDs adhere to the minimum constraints for these items. However, it does not check that the public ID follows the standard conventions for public identifiers; it does not check that the system ID is a legal URL; and it does not even check the characters in the internal DTD subset, much less the syntax.

As an example of this class, let’s look at a program that validates XHTML 1.0 documents. XHTML validity is a little stricter than HTML validity. In particular, according to the XHTML 1.0 specification, a valid XHTML document must satisfy these four conditions:

  • It must be valid according to one of the three XHTML DTDs: strict, transitional, or frameset.

  • The root element of the document must be html.

  • This root html element of the document must specify the default namespace as http://www.w3.org/1999/xhtml using an xmlns attribute.

  • The document must contain a DOCTYPE declaration. The public identifier for the external DTD subset must reference one of the three XHTML DTDs using one of these three public identifiers:

    • -//W3C//DTD XHTML 1.0 Strict//EN

    • -//W3C//DTD XHTML 1.0 Transitional//EN

    • -//W3C//DTD XHTML 1.0 Frameset//EN

There are a few other flaky rules scattered throughout the XHTML specification, mostly involving constraints that can’t be reasonably specified in a DTD such as that an a element cannot contain another a element, but these are the major ones that define strict XHTML conformance.

Example 15.20 is similar to the earlier JDOMValidator. That is, it reads a URL from the command line and validates the document found at that URL against its DTD. However, it also checks the above constraints. Of particular interest for this section is that it checks that the document type declaration is pointing to one of the three legal DTDs. This is something pure XML validation normally doesn’t tell you.

Example 15.20. Validating XHTML with the DocType class

import java.io.IOException;
import org.jdom.*;
import org.jdom.input.SAXBuilder;

public class XHTMLValidator {

  public static void main(String[] args) {

    for (int i = 0; i < args.length; i++) {
      validate(args[i]);
    }

  }

  private static SAXBuilder builder = new SAXBuilder(true);
                               /* turn on validation ^^^^ */

  // not thread safe
  public static void validate(String source) {

      Document document;
      try {
        document = builder.build(source);
      }
      catch (JDOMException e) {
        System.out.println(source
         + " is invalid XML, and thus not XHTML.");
        return;
      }
      catch (IOException e) {
        System.out.println("Could not read: " + source);
        return;
      }

      // If we get this far, then the document is valid XML.
      // Check to see whether the document is actually XHTML
      boolean valid = true;
      DocType doctype = document.getDocType();

      if (doctype == null) {
        System.out.println("No DOCTYPE");
        valid = false;
      }
      else {
        // verify the DOCTYPE
        String name     = doctype.getElementName();
        String systemID = doctype.getSystemID();
        String publicID = doctype.getPublicID();

        if (!name.equals("html")) {
          System.out.println(
           "Incorrect root element name " + name);
          valid = false;
        }

        if (publicID == null
         || (!publicID.equals("-//W3C//DTD XHTML 1.0 Strict//EN")
           && !publicID.equals(
            "-//W3C//DTD XHTML 1.0 Transitional//EN")
           && !publicID.equals(
            "-//W3C//DTD XHTML 1.0 Frameset//EN"))) {
          valid = false;
          System.out.println(source
           + " does not seem to use an XHTML 1.0 DTD");
        }
      }

      // Check the namespace on the root element
      Element root = document.getRootElement();
      Namespace namespace = root.getNamespace();
      String prefix = namespace.getPrefix();
      String uri = namespace.getURI();
      if (!uri.equals("http://www.w3.org/1999/xhtml")) {
        valid = false;
        System.out.println(source
         + " does not properly declare the"
         + " http://www.w3.org/1999/xhtml namespace"
         + " on the root element");
      }
      if (!prefix.equals("")) {
        valid = false;
        System.out.println(source
         + " does not use the empty prefix for XHTML");
      }

      if (valid) System.out.println(source + " is valid XHTML.");

  }

}

Here’s the result of running this program on the XHTML 1.0 specification:

D:\books\XMLJAVA>java XHTMLValidator http://www.w3.org/TR/xhtml1/
http://www.w3.org/TR/xhtml1/ is valid XHTML.

As one would hope, it proves valid.

Prev | Up | Next
Namespaces | Home | The EntityRef Class
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified August 11, 2002


The EntityRef Class
Prev Chapter 15. The JDOM Model Next

The EntityRef Class

The EntityRef class shown in Example 15.21 represents a defined entity reference such as &copy; or &chapter1;. It is used only for entity references the parser does not expand. Given a fully validating parser, or even just one that reads the external DTD subset, no EntityRef objects will normally be present in the tree.

Example 15.21. The JDOM EntityRef class

package org.jdom;

public class EntityRef implements Serializable, Cloneable {

  protected String name;
  protected String publicID;
  protected String systemID;
  protected Object parent;

  protected EntityRef();

  public EntityRef(String name);
  public EntityRef(String name, String systemID);
  public EntityRef(String name, String publicID,
   String systemID);

  public EntityRef detach();

  public Document  getDocument();
  public String    getName();
  public EntityRef setName(String name);
  public Element   getParent();
  public String    getPublicID();
  public EntityRef setPublicID(String newPublicID);
  public String    getSystemID();
  public EntityRef setSystemID(String newSystemID);

  public final boolean equals(Object ob);
  public final int     hashCode();
  public       String  toString();
  public       Object  clone();

}

Each EntityRef object has these four properties:

  • The entity name

  • The public ID of the entity

  • The system ID of the entity

  • The parent Element of the entity

The public and system IDs will be null if the parser did not read the part of the DTD that defined the entity.

The one thing you might expect that is not available is the entity’s replacement text. Unlike the EntityReference interface in DOM, JDOM EntityRef objects do not have any children. If the builder knows the replacement text of the entity, it will insert the corresponding nodes in the tree rather than including an EntityRef object.

There’s not a frequent need to use this class directly. You can add an entity reference in place of characters you know are going to cause problems in your choice of encoding. However, you’re probably better off just letting the XMLOutputter emit numeric character references instead. If you do choose to insert EntityRef objects into your JDOM tree, then be sure to use a DocType that either points to an external DTD subset or includes an internal DTD subset that defines your entities. JDOM will not do this for you automatically, so if you aren’t careful you can produce a malformed document.

For an example, let’s turn once again to XHTML. Browsers generally use non-validating parsers and tend not to read the external DTD subset by default. Thus they’re likely to encounter skipped entity references. Therefore, the XHTML specification states:

If it encounters an entity reference (other than one of the predefined entities) for which the User Agent has processed no declaration (which could happen if the declaration is in the external subset which the User Agent hasn’t read), the entity reference should be rendered as the characters (starting with the ampersand and ending with the semi-colon) that make up the entity reference.

Here is a simple method that assists with this requirement by converting all EntityRef objects in a tree to Text objects of the form &name;.

  public static void entityRefToText(Element element) {

    List content = element.getContent();
    ListIterator iterator = content.listIterator();
    while (iterator.hasNext()) {
      Object o = iterator.next();
      if (o instanceof Element) {
        Element child = (Element) o;
        entityRefToText(child);
      }
      else if (o instanceof EntityRef) {
        EntityRef ref = (EntityRef) o;
        Text fauxRef = new Text("&");
        fauxRef.append(ref.getName());
        fauxRef.append(";");
        iterator.set(fauxRef);
      }
    }

  }

There’s one technique here you may not have seen before. Instead of a basic Iterator I used a ListIterator. The reason is that ListIterator has an optional set() method (which JDOM does implement) that replaces the last object returned by next() with another object. That’s how I replace the EntityRef with a Text.

Caution

Be sure you understand the difference here. A Text object always contains plain text, never an entity reference or a tag, even if it contains some characters like & and < that might need to be escaped when the document is serialized. For instance, invoking element.setText("&amp;") sets the content of element to the five characters &, a, m, p, and ; in that order. It does not set it to the single character &.

Prev | Up | Next
The DocType class | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 04, 2002


Summary
Prev Chapter 15. The JDOM Model Next

Summary

JDOM models an XML document as a Document object which contains a List of Comment and ProcessingInstruction objects and a single Element object for the root element.

Each Element object contains a List of its children: Comment, ProcessingInstruction, Text, and other Element objects. In addition, it has separate lists of attributes and additional namespaces. The list of children is accessible through the getContent() method. Content can be appended to an element or to a document with the heavily overloaded addContent() method or removed from the tree using the heavily overloaded removeContent() method. However, other mutation operations such as inserting a node at the beginning or in the middle of an element’s children require using the methods of java.util.List.

With the exception of Namespace, the JDOM core node classes have public constructors, and are constructed in about the most obvious ways you can imagine. For instance, to create a new Attribute object that represents the attribute revision="20020430", you’d use the constructor Attribute("revision", "20020430"). Namespaces and local names are always separated in argument lists. Sometimes namespaces are provided as one or two strings containing the namespace URI and the optional prefix. Other times they’re provided as a combined Namespace object. (JDOM’s not perfectly consistent here.)

All the core node classes have the getter and setter methods you’d expect; e.g. setTarget(), getTarget(), setData(), and getData() for ProcessingInstruction; setName(), getName(), setNamespace(), and getNamespace() for Element and so forth. In addition they’re a few common methods that show up across several classes. The detach() method removes a node from its parent. The getParent() method returns a node’s parent element. And all the JDOM nodes provide the usual Java utility methods equals(), hashCode(), and toString(). Finally, all these classes implement both Serializable and Cloneable and; all override clone().

Prev | Up | Next
The EntityRef Class | Home | Chapter 16. XPath
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 03, 2002


Chapter 16. XPath
Prev Next

Chapter 16. XPath

Table of Contents

Queries
The XPath Data Model
Location Paths
Axes
Node tests
Predicates
Compound Location Paths
Absolute Location Paths
Abbreviated Location paths
Combining location paths
Expressions
Literals
Operators
Functions
XPath Engines
XPath with Saxon
XPath with Xalan
DOM Level 3 XPath
Namespace Bindings
Snapshots
Compiled Expressions
Jaxen
Summary

Much of the code in this book has involved navigating the tree structure of an XML document to find particular nodes. For example, the XML-RPC servlet in Chapter 10 read a client request looking for int elements. Such code can become quite involved and fragile if you aren’t very careful. As the code walks down the tree hierarchy, loading one child after the other, a single misplaced or misnamed element may cause the program to fail. If an element isn’t where it’s expected to be, the chain of method calls that gives directions to the desired elements will be broken. What’s needed is a way to specify which nodes a program needs without explicitly specifying how the program navigates to those nodes.

XPath is a fourth generation declarative language for locating nodes in XML documents. An XPath location path says which nodes from the document you want. It says nothing about what algorithm is used to find these nodes. You simply pass an XPath statement to a method, and the XPath engine is responsible for figuring out how to find all the nodes satisfying that expression. This is much more robust than writing the detailed search and navigation code yourself using DOM, SAX, or JDOM. XPath searches often succeed even when the document format is not quite what you expected. For example, a comment in the middle of a paragraph of text may break DOM code that expects to see contiguous text. XPath wouldn’t be phased by this. Many XPath expressions are resistant even to much more significant alterations such as changing the names or namespaces of ancestor elements, reordering the children of an element, or even adding or subtracting entire levels from the tree hierarchy.

In the large, using XPath in a Java program is like using SQL in a Java program. To extract information from a database, you write a SQL statement indicating what information you want and you ask JDBC to fetch it for you. You neither know nor care how JDBC communicates with the database. Similarly with XML, you write an XPath expression indicating what information you want from an XML document and ask the XPath engine to fetch it, without concerning yourself with the exact algorithms used to search the XML document.

Queries

XPath can be thought of as a query language like SQL. However, rather than extracting information from a database, it extracts information from an XML document. An example should help make this more concrete. Consider the simple weather report document in Example 16.1.

Example 16.1. Weather data in XML

<?xml version="1.0" encoding="ISO-8859-1"?>
<weather time="2002-06-06T15:35:00-05:00">
  <report latitude="41.2° N" longitude="71.6° W">
    <locality>Block Island</locality>
    <temperature units="°C">16</temperature>
    <humidity>88%</humidity>
    <dewpoint units="°C">14</dewpoint>
    <wind>
      <direction>NE</direction>
      <speed units="km/h">16.1</speed>
      <gust units="km/h">31</gust>
    </wind>
    <pressure units="hPa">1014</pressure>
    <condition>overcast</condition>
    <visibility>13 km</visibility>
  </report>
  <report latitude="34.1° N" longitude="118.4° W">
    <locality>Santa Monica</locality>
    <temperature units="°C">19</temperature>
    <humidity>79%</humidity>
    <dewpoint units="°C">16</dewpoint>
    <wind>
      <direction>WSW</direction>
      <speed units="km/h">14.5</speed>
    </wind>
    <pressure units="hPa">1010</pressure>
    <condition>hazy</condition>
    <visibility>5 km</visibility>
  </report>
</weather>

Here are some XPath expressions that identify particular parts of this document:

  • /weather/report is an XPath expression that selects the two report elements.

  • /weather/report[1] is an XPath expression that selects the first report element.

  • /weather/report/temperature is an XPath expression that selects the two temperature elements.

  • /weather/report[locality="Santa Monica"] is an XPath expression that selects the second report element.

  • //report[locality="Block Island"]/attribute::longitude is an XPath expression that selects the longitude attribute of the first report element.

  • /child::weather/child::report/child::wind/child::* is an XPath expression that selects all the direction, speed, and gust elements.

  • 9 * number(/weather/report[locality="Block Island"]/temperature) div 5 + 32 is an XPath expression that returns the temperature on Block Island in degrees Fahrenheit.

  • /descendant::* is an XPath expression that selects all the elements in the document.

Like SQL, XPath expressions are used in many different contexts including:

  • Dedicated query tools like Alex Chaffee’s XPath Explorer. Figure 16.1 shows this tool evaluating the expression /weather/report/temperature against Example 16.1.

  • Native XML databases like the Apache XML Project’s XIndice and Software AG’s Tamino.

  • As a component of other, broader languages like XSLT and XQuery.

  • And last and certainly not least, as a search component for your own Java programs that read XML documents.

Figure 16.1. XPath Explorer

Prev | Up | Next
Summary | Home | The XPath Data Model
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 02, 2002


The XPath Data Model
Prev Chapter 16. XPath Next

The XPath Data Model

An XPath query operates on a namespace well-formed XML document after it has been parsed into a tree structure. The particular tree model XPath uses divides each XML document into seven kinds of nodes:

root node

The document itself. The root node’s children are the comments and processing instructions in the prolog and epilog and the root element of the document.

element node

An element. Its children are all the child elements, text nodes, comments, and processing instructions the element contains. An element also has namespaces and attributes. However, these are not child nodes.

attribute node

An attribute other than one that declares a namespace

text node

The maximum uninterrupted run of text between tags, comments, and processing instructions. White space is included.

comment node

A comment

processing instruction node

A processing instruction

namespace node

A namespace mapping in scope on an element

The XPath data model does not include entity references, CDATA sections, or the document type declaration. Entity references are resolved into their component text and elements. CDATA sections are treated like any other text, and will be merged with any adjacent text before a text node is formed. Default attributes are applied, but otherwise the document type declaration is not considered.

In the XPath data model each node has a string-value Furthermore, attributes, elements, processing instructions, and namespace nodes have expanded names, which are divided into a local part and a namespace URI. Table 16.1 summarizes XPath’s rules for calculating names and values for its seven node types.

Table 16.1. XPath Expanded Names and String-values

Node type | Local name | Namespace name | String-value
root | None | None | the complete, ordered content of all text nodes in the document; same as the value of the root element of the document
element | The name of the element, not including any prefix or colon | The namespace URI of the element | The complete, ordered content of all text node descendants of this element (i.e. the text that’s left after all references are resolved and all other markup is stripped out.)
attribute | The name of the attribute, not including any prefix or colon | The namespace URI of the attribute | The normalized attribute value
text | None | None | The complete content of the text node
processing instruction | The target of the processing instruction | None | The processing instruction data
comment | None | None | The text of the comment
namespace | The prefix for the namespace | None | The absolute URI for the namespace

If an XPath function such as local-name() or namespace-uri() attempts to retrieve the value of one of these properties for a node that doesn’t have that property, it returns the empty string.

An diagram and example should help explain this. Consider the simple SOAP response document in Example 16.2.

Example 16.2. A SOAP response document

<?xml version="1.0"?>
<!-- XPath data model example -->
<SOAP-ENV:Envelope
  xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" />
  <SOAP-ENV:Body>
    <Quote
      xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
      <Price currency="USD">4.12</Price>
    </Quote>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Figure 16.2 is a UML object diagram that identifies the connections between and properties of the different XPath nodes in this document. Solid lines indicate a child relationship. Dashed and dotted lines indicate namespace and attribute connections respectively. Document order runs from top to bottom and left-to-right (although the exact order of namespace nodes and attribute nodes attached to the same element is implementation dependent). One thing to note is that white space is significant in the XPath data model. Line breaks are indicated by \n in this figure.

Figure 16.2. An XPath data model

The differences between the XPath and DOM data models

The XPath data model is similar to, but not quite the same as the DOM data model. The most important differences relate to the names and values of` nodes. In XPath, only attributes, elements, processing instructions, and namespace nodes have names, which are divided into a local part and a namespace URI. XPath does not use pseudo-names like #document and #comment. The other big difference is that in XPath the value of an element or root node is the concatenation of the values of all its text node descendants, not null as it is in DOM. For example, the XPath value of <p>Hello</p> is the string Hello and the XPath value of <p>Hello<em>Goodbye</em></p> is the string HelloGoodbye.

Other differences between the DOM and XPath data models include:

  • XPath does not have separate nodes for CDATA sections. CDATA sections are simply merged with their surrounding text.

  • XPath does not include any representation of the document type declaration.

  • Each XPath text node always contains the maximum contiguous run of text. No text node is adjacent to any other text node.

  • All entity references must be resolved before an XPath data model can be built. Once resolved they are not reported separately from their contents.

  • In XPath, the element that contains an attribute is the parent of that attribute, although the attribute is not a child of the element.

  • Every namespace which has scope for an element or attribute is an XPath namespace node for that element or attribute. This does not refer to namespace declaration attributes such as xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#", but rather to all elements for which a namespace mapping is defined. There are no nodes in an XPath model that directly represent namespace declaration attributes.

Prev | Up | Next
Chapter 16. XPath | Home | Location Paths
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 07, 2002


Location Paths
Prev Chapter 16. XPath Next

Location Paths

Although there are many different kinds of XPath expressions, the one that’s of primary use in Java programs is the location path. A location path selects a set of nodes from an XML document. Each location path is composed of one or more location steps. Each location step has an axis, a node test and, optionally, one or more predicates. Furthermore, each location step is evaluated with respect to a particular context node. A double colon (::) separates the axis from the node test, and each predicate is enclosed in square brackets.

Some examples will help explain all these terms. Consider the simple XML-RPC request document in Example 16.3.

Example 16.3. An XML-RPC request document

<?xml version="1.0"?>
<methodCall>
  <methodName>calculateFibonacci</methodName>
  <params>
    <param>
      <value>
        <int>23</int>
      </value>
    </param>
  </params>
</methodCall>

Exactly how the context node for a location step is determined depends on the environment in which the location step appears. When using XPath in Java code, you normally pass the context node as an argument to the method that evaluates the expression. In XSLT the context node is normally the currently matched node in the input document. In other environments, other means are provided to choose the context node. For now, let’s just pick the root methodCall element as the context node. Then child::methodName is a location step that selects a node-set containing the single methodName element. It moves along the child axis with the node test methodName. That is, it selects all the children of the context node named methodName. child::params returns a node-set containing the single params element.

Location paths are not guaranteed to return a node-set that contains exactly one node (and assuming they do is a very common mistake). child::* returns a node-set containing two element nodes, one for the methodName element and one for the params element. The asterisk is a wild card node test that matches any element, regardless of name.

Axes

There are twelve axes along which a location step can move. Each selects a different subset of the nodes in the document, depending on the context node. These are:

self

The node itself.

child

All child nodes of the context node. (Attributes and namespaces are not considered to be children of the node they belong to.)

descendant

All nodes completely contained inside the context node (between the end of its start-tag and the beginning of its end-tag); that is, all child nodes, plus all children of the child nodes, and all children of the children’s children, and so forth. This axis is empty if the context node is not an element node or a root node.

descendant-or-self

All descendants of the context node and the context node itself.

parent

The node which most immediately contains the context node. The root node has no parent. The parent of the root element and comments and processing instructions in the document’s prolog and epilog is the root node. The parent of every other node is an element node. The parent of a namespace or attribute node is the element node that contains it, even though namespaces and attributes aren’t children of their parent elements.

ancestor

The root node and all element nodes that contain the context node.

ancestor-or-self

All ancestors of the context node and the context node itself.

preceding

All non-attribute, non-namespace nodes which come before the context node in document order and which are not ancestors of the context node

preceding-sibling

All non-attribute, non-namespace nodes which come before the context node in document order and have the same parent node

following

All non-attribute, non-namespace nodes which follow the context node in document order and which are not descendants of the context node.

following-sibling

All non-attribute, non-namespace nodes which follow the context node in document order and have the same parent node

attribute

Attributes of the context node. This axis is empty if the context node is not an element node.

namespace

Namespaces in scope on the context node. This axis is empty if the context node is not an element node.

For example, consider the slightly more complex SOAP request document in Example 16.4. Let us pick the middle Quote element (the one whose symbol is AAPL) as the context node and move along each of the axes from there.

Example 16.4. A SOAP request document

<?xml version="1.0"?>
<!-- XPath axes example -->
<SOAP-ENV:Envelope
  xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns="http://namespaces.cafeconleche.org/xmljava/ch2/">
  <SOAP-ENV:Body>
    <Quote symbol="RHAT">
      <Price currency="USD">7.02</Price>
    </Quote>
    <Quote symbol="AAPL">
      <Price currency="USD">24.85</Price>
    </Quote>
    <Quote symbol="BAC">
      <Price currency="USD">68.59</Price>
    </Quote>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

  • The self axis contains one node, the middle Quote element that was chosen as the context node.

  • The child axis contains three nodes: a text node containing white space, an element node with the local name Price, and another text node containing only white space, in that order. (All the white space counts, though there are ways to get rid of it or ignore it if you want to, as you’ll see later.)

  • The descendant axis contains four nodes: a text node containing white space, an element node with the local name Price, a text node with the value "24.85", and another text node containing only white space, in that order.

  • The descendant-or-self axis contains five nodes: an element node with the local name Quote, a text node containing white space, an element node with the local name Price, a text node with the value "24.85", and another text node containing only white space, in that order.

  • The parent axis contains a single element node with the local name Body.

  • The ancestor axis contains three nodes: an element node with the local name Body, an element node with the local name Envelope, and the root node in that order.

  • The ancestor-or-self axis contains four nodes: an element node with the local name Quote, an element node with the local name Body, an element node with the local name Envelope, and the root node in that order.

  • The preceding axis contains eight nodes: a text node containing only white space, another text node containing only white space, a text node containing the string 7.02, an element node named Price, another text node containing only white space, an element node named Quote, a text node containing only white space, a comment node in that order. Note that ancestor elements and attribute and namespace nodes are not counted along the preceding axis.

  • The preceding-sibling axis contains three nodes: a text node containing white space, an element node with the name Quote and the symbol RHAT, and another text node containing only white space.

  • The following axis contains eight nodes: a text node containing only white space, a Quote element node, a text node containing only white space, a Price element node, a text node containing the string 68.59, and three text nodes containing only white space. Descendants are not included in the following axis.

  • The following-sibling axis contains three nodes: a text node containing white space, an element node with the name Quote and the symbol BAC, and another text node containing only white space.

  • The attribute axis contains one attribute node with the name symbol and the value AAPL.

  • The namespace axis contains two namespace nodes, one with the name SOAP-ENV and the value http://schemas.xmlsoap.org/soap/envelope/ and the other with an empty string name and the value http://namespaces.cafeconleche.org/xmljava/ch2/.

Generally these sets would be further subsetted via a node test. For example, if the location step preceding::Quote were applied to this context node, then the resulting node-set would only contain a single node, an element node named Quote.

Node tests

The axis chooses the direction to move from the context node. The node test determines what kinds of nodes will be selected along that axis. The node tests are:

Name

Any element or attribute with the specified name. If the name is prefixed, then the local name and namespace URI are compared, not the qualified names. If the name is not prefixed, then the element must be in no namespace at all. An unprefixed name in an XPath expression never matches an element in a namespace, even in the default namespace. When using XPath to search for an unprefixed element like Quote that is in a namespace, you have to use a prefixed name instead such as stk:Quote. Exactly how the prefix is mapped to the namespace depends on the environment in which the XPath expression is used.

*

Along the attribute axis the asterisk matches all attribute nodes. Along the namespace axis the asterisk matches all namespace nodes. Along all other axes, this matches all element nodes.

prefix:*

Any element or attribute in the namespace mapped to the prefix.

comment()

Any comment

text()

Any text node

node()

Any node

processing-instruction()

Any processing instruction

processing-instruction('target')

Any processing instruction with the specified target

For example, once again considering the SOAP request document in Example 16.4 and choosing the AAPL Quote element as the context node, consider these location steps:

  • self::* selects one node, the middle Quote element that serves as the context node.

  • child::* selects one node, an element node with the name Price and the value 24.85.

  • child::Price selects no nodes because there are no Price elements in this document that are not in any namespace.

  • child::stk:Price selects one node, an element node with the name Price and the value 24.85, provided that the prefix stk is bound to the http://namespaces.cafeconleche.org/xmljava/ch2/ namespace URI in the local environment.

  • descendant::text() selects three nodes: a text node containing white space, a text node with the value "24.85", and another text node containing only white space.

  • descendant-or-self::* selects two nodes: an element node with the name Quote and an element node with the name Price.

  • parent::SOAP-ENV:Envelope selects an empty node set because the parent of the context node is not SOAP-ENV:Envelope.

  • ancestor::SOAP-ENV:Envelope selects one node, the document element, assuming that the local environment maps the prefix SOAP-ENV to the namespace URI http://schemas.xmlsoap.org/soap/envelope/.

  • ancestor::SOAP-ENV:* selects two nodes, the SOAP-ENV:Body element and the SOAP-ENV:Envelope element, again assuming that the prefixes are properly mapped.

  • ancestor-or-self::* selects three nodes: an element node with the local name Quote, an element node with the local name Body, and an element node with the local name Envelope.

  • preceding::comment() selects the single comment in the prolog.

  • preceding-sibling::node() selects three nodes: a text node containing white space, an element node with the name Quote and the symbol RHAT, and another text node containing only white space, in that order.

  • following::* selects two nodes: a Quote element node and a Price element node.

  • following-sibling::processing-instruction() returns an empty node-set.

  • attribute::symbol selects the attribute node with the name symbol and the value AAPL.

  • namespace::SOAP-ENV returns a node-set containing a namespace node with name SOAP-ENV and the value http://schemas.xmlsoap.org/soap/envelope/.

  • namespace::* returns a node-set containing two namespace nodes, one with the name SOAP-ENV and the value http://schemas.xmlsoap.org/soap/envelope/ and the other with an empty string name and the value http://namespaces.cafeconleche.org/xmljava/ch2/.

Predicates

Each location step can have zero or more predicates that further filter the node-set. A predicate is an XPath expression in square brackets that is evaluated for each node selected by the location step. If the predicate is true, then the node is kept in the node-set. If the predicate is false, then the node is removed from the node-set. For example, given the same SOAP request document, suppose the context node is now the SOAP-ENV:Body element and that the stk prefix is mapped to the http://namespaces.cafeconleche.org/xmljava/ch2/ namespace URI. This location step returns a node-set containing all the Quote elements whose price is less than ten:

child::stk:Quote[child::stk:Price < 10]

If this XPath expression were embedded in an XML document, you might need to escape the less than sign as &lt;. However, this is not necessary when using XPath expressions in Java programs.

There can be more than one predicate. For example, this location step checks both that the absolute price is greater than ten and that the currency is U.S. dollars:

child::stk:Quote[child::stk:Price > 10][attribute::currency = "USD"]

If the predicate returns a number, then the node is kept in the set only if the number is equal to the position of the context node in the context node list. For example, this location step selects the third Quote child of the context node but not the first or second:

child::stk:Quote[3]

If the context node has fewer than three Quote children, then this returns an empty node-set.

If the predicate returns a string, then the context node is deleted from the set if the string is empty and kept otherwise. For example, this location step selects those Quote elements whose symbol attribute has a value:

child::stk:Quote[string(attribute::symbol)]

This is not quite the same as selecting the Quote elements that have a symbol attribute. This Quote element would not be matched by the above location step:

    <Quote symbol="">
      <Price currency="USD">17.32</Price>
    </Quote>

If the predicate returns a node-set, then the source node is kept in the returned set only if the predicate node-set is non-empty. It is deleted otherwise. For example, this location step finds those Quote children of the context node that have at least one Price child:

child::stk:Quote[child::stk:Price]

This location step finds those Quote children of the context node that have at least one Price child and at least one Quantity child:

child::stk:Quote[child::stk:Price][child::stk:Quantity]

When applied to the SOAP-ENV:Body element in Example 16.4, it returns an empty node-set because none of its Quote children have a Quantity child.

Compound Location Paths

The forward slash (/) combines location steps into a location path. The node-set selected by the first step becomes the context node-set for the second step. The node-set identified by the second step becomes the context node-set for the third step, and so on.

Continuing with the same example in Example 16.4 and still using the second Quote element as the context node, consider these location paths (Here I assume that the environment for the XPath expressions maps the prefix stk to the namespace URI http://namespaces.cafeconleche.org/xmljava/ch2/ and the prefix SOAP-ENV to the namespace URI http://schemas.xmlsoap.org/soap/envelope/):

child::stk:Price/attribute::currency

This selects the currency attribute node currency="USD"

preceding-sibling::stk:Quote/descendant::*

This selects one node, the first value element in the document.

parent::*/child::stk:Quote

This selects all three Quote element nodes in the document, including the context node itself.

parent::*/child::stk:Quote[child::stk:Price > 20]

This selects the AAPL and the BAC Quote element nodes, but not the RHAT Quote element node.

parent::*/descendant::stk:Price

This selects all three Price element nodes in the document.

parent::*/child::stk:Quote[attribute::symbol='BAC']/child::stk:Price

This selects the Price element node of the BAC Quote element.

parent::*/descendant::stk:Price/attribute::currency

This selects all three currency attribute nodes in the document.

Absolute Location Paths

So far all the location paths have been relative to a specified context node. To date, I’ve just identified that context node in prose. When we begin discussing XPath APIs, you’ll see that most methods for evaluating an XPath expression have a context node argument. However, not all location paths require context nodes. In particular, a location path that begins with a forward slash (/) is an absolute path that starts at the root node of the document (not the root element but the root node).

Continuing with the same example in Example 16.4 and once again assuming that the environment binds the prefix stk to the namespace URI http://namespaces.cafeconleche.org/xmljava/ch2/ and the prefix SOAP-ENV to the namespace URI http://schemas.xmlsoap.org/soap/envelope/, consider these location paths:

/child::SOAP-ENV:Envelope/child::SOAP-ENV:Body/child::stk:Quote/child::stk:Price

This selects all three Price element nodes.

/child::SOAP-ENV:Envelope/child::SOAP-ENV:Body

This selects the single SOAP-ENV:Body element node.

/descendant::stk:Price

This selects all three Price element nodes in the document.

/descendant::stk:Quote[child::stk:Price > 20]

This selects the Quote element nodes whose Price is greater than 20; i.e. it selects the AAPL and the BAC Quote element nodes, but not the RHAT Quote element node.

/child::SOAP-ENV:Body

This returns an empty node-set because the root element of the document is SOAP-ENV:Envelope, not SOAP-ENV:Body.

/descendant::*/attribute:*

This returns a node-set containing all attribute nodes in the document.

/descendant-or-self::node()

This returns a node-set containing all non-attribute, non-namespace nodes in the document.

/

This selects the root node of the document.

Abbreviated Location paths

XPath location paths can use the abbreviations listed in Table 16.2 in location paths. The semantics are the same. The syntax is just a little easier to type.

Table 16.2. Abbreviated syntax for XPath

Abbreviation | Expanded form
Name | child::Name
@Name | attribute::Name
// | /descendant-or-self::node()/
. | self::node()
.. | parent::node()

Using the abbreviated forms, the previous batch of relative XPaths selecting from Example 16.4 using the second Quote element as the context node can be rewritten like this:

stk:Price/@currency

This selects the currency attribute node currency="USD"

preceding-sibling::stk:Quote//*

This isn’t an exact abbreviation for preceding-sibling::stk:Quote/descendant::* (// expands to /descendant-or-self::node()/, not /descendant::) but the node-set selected is the same, the first Price element in the document.

../stk:Quote

This selects all three Quote element nodes in the document, including the context node itself.

..//stk:Price

This also isn’t an exact abbreviation for the original expression, but again it selects the same node-set, which in this case contains all three Price element nodes in the document.

../stk:Quote[stk:Price > 20]

This selects the AAPL and the BAC Quote element nodes, but not the RHAT Quote element node.

../stk:Quote[@symbol='BAC']/stk:Price

This selects the Price child element node of the BAC Quote element.

..//stk:Price/@currency

This too isn’t an exact abbreviation for the original expression, but once again it selects the same node-set containing all three currency attribute nodes in the document.

Absolute location paths can also be abbreviated. In this case // is especially convenient because at the start of a location path it produces a node-set containing every non-attribute, non-namespace node in the document. However, you should be warned that it is quite inefficient in most XPath processors. If it’s possible to rewrite an expression so that it does not use // (or the unabbreviated descendant or descendant-or-self axes), you probably should.

Here are some example of abbreviated absolute location paths that apply to Example 16.4:

/SOAP-ENV:Envelope/SOAP-ENV:Body/stk:Quote

This selects all three Quote element nodes.

/SOAP-ENV:Envelope/SOAP-ENV:Body

This selects the single SOAP-ENV:Body element node.

//stk:Price

This selects all three Price element nodes in the document.

//stk:Quote[stk:Price > 20]

This selects the Quote element nodes whose Price is greater than 20; i.e. it selects the AAPL and the BAC Quote element nodes, but not the RHAT Quote element node.

/stk:Price

This returns an empty node-set because the root element of the document is SOAP-ENV:Envelope, not Price.

//@*

This returns a node-set containing all attribute nodes in the document.

//.

This returns a node-set containing all non-attribute, non-namespace nodes in the document.

Combining location paths

Occasionally it’s useful to select a node-set that’s built from multiple, more or less unrelated parts of an XML document. For example, you might want to select all the Price elements and all the Quote elements in a document. //stk:Price selects all the prices. //stk:Quote selects all the quotes. You can use the vertical bar, |, to combine these two node-sets into one.

//stk:Price | //stk:Quote

selects all the Price element nodes and all the Quote element nodes in the document.

//@currency | //stk:Price

selects all the currency attribute nodes and all the Price element nodes.

//stk:Quote/stk:Price | //stk:Quote/stk:Quantity

selects all the Price and Quantity child elements of all Quote elements.

Prev | Up | Next
The XPath Data Model | Home | Expressions
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 07, 2002


Expressions
Prev Chapter 16. XPath Next

Expressions

Not all XPath expressions are location paths. In fact, you’ve already seen several that weren’t. The content of the square brackets in a location step predicate is a more generic form of XPath expression. Each XPath 1.0 expression returns one of these four types:

string

A sequence of zero or more Unicode characters. This is not quite the same thing as a Java String which is a sequence of UTF-16 code points. A single Unicode character from outside Unicode’s Basic Multilingual Plane (BMP) occupies two UTF-16 code points. XPath strings that contain characters from outside the BMP will have smaller lengths than the equivalent Java string.

number

An IEEE-754 double. This is the same as Java’s double primitive data type for all intents and purposes.

boolean

Semantically the same as Java’s boolean type. However, XPath does allow 1 and 0 to represent true and false respectively.

node-set

An unordered collection of nodes from an XML document without any duplicates. Since a node-set is a mathematical set, there is no fundamental ordering defined on the set. However, most node-sets have a natural document order that’s derived from the order of the nodes in the set in the input document. This is similar to how the set of integers {1, 4, -76, 23} is unordered. However, the individual elements in the set can be compared to each other and sorted if desired. In practice, most APIs use lists rather than sets to represent node-sets, and these lists are sorted in either document order or reverse document order, depending on how they were created.

Different XPath engines map these four types to different Java classes and primitive data types. For example, Jaxen uses the normal Java classes Boolean, List, Double, and String, whereas jd.xslt uses the custom types XBoolean, XNodeSet, XNumber, and XString; and DOM3 XPath uses a single XPathResult interface that can hold any of the four XPath types. In all cases, there are straightforward methods to convert these to the usual Java primitive types like boolean and double.

The XPath expression syntax includes literal forms for strings and numbers as well as operators and functions for manipulating all four XPath data types.

The primary use-case for XPath literals and operators is predicates. Although you can use these to perform simple arithmetic and string operations with XPath expressions, you’re more likely to do complex work of this sort in the Java code. However, the functions can perform some very useful operations on node-sets that would be much harder to implement in SAX, DOM, or JDOM.

Literals

XPath defines literal forms for strings and numbers. Numbers have more or less the same form as double literals in Java. That is, they look like 72.5, -72.5, .5321, and so forth. XPath only uses floating point arithmetic, so integers like 42, -23, and 0 are also number literals. However, XPath does not recognize scientific notation such as 5.5E-10 or 6.022E23.

XPath string literals are enclosed in single or double quotes. For example, "red" and 'red' are different representations for the same string literal containing the word red.

There are no boolean or node-set literals. However, the true() and false() functions sometimes substitute for the lack of boolean literals.

Operators

XPath provides the following operators for basic floating point arithmetic:

+ | addition
- | subtraction
* | multiplication
div | division
mod | taking the remainder

All five behave the same as the equivalent operators in Java. The keywords div and mod are used instead of / and % respectively.

XPath also provides these operators for comparisons and boolean logic:

< | less than
> | greater than
<= | less than or equal to
>= | greater than or equal to
= | boolean equals (not an assignment statement as in Java)
!= | not equal to
or | Boolean or
and | Boolean and

In an XML context such as an XSLT stylesheet, some of these may need to be escaped with &lt; or &gt;. However, this is normally not necessary when using XPath in Java code.

Additional arithmetic and boolean operations such as rounding and negation are provided by various XPath functions.

Functions

XPath defines a number of useful functions that operate on and return the four fundamental XPath data types. Some of these take variable numbers of arguments. In the list below, optional arguments are suffixed with a question mark. A function that doesn’t have any arguments normally operates on the context node instead. For the most part these functions are weakly typed. You can pass any of the four types in the place of an argument that is declared to be of type boolean, number, or string. XPath will convert it and use it. The exceptions are those functions that are declared to take node-sets as arguments. XPath cannot convert arguments of other types to node-sets.

None of these functions modify their arguments in anyway. An object passed to any of these functions will be the same after the function returns as it was before the function was invoked. However, many of these functions return a new object which is a variant of one of the arguments. This characteristic is necessary to make XSLT (which depends on XPath) a functional language.

Node-set functions

number last()

Returns the number of nodes in the context node list. This is the same as the position of the last node in the list.

number position()

Returns the position of the context node in the context node list. The first node has position 1, not 0.

number count(node-set)

Returns the number of nodes in the argument

node-set id(object)

Returns a node-set containing the single element node with the specified id as determined by an ID-type attribute. If no node has the specified ID, then this function returns an empty node-set. If the argument is a node-set, then it returns a node-set containing all the element nodes whose ID matches the string-value of any of the nodes in the argument node-set.

string local-name(node-set?)

Returns the local name of the first node in the argument node-set, or the local name of the context node if the argument is omitted. It returns an empty string if the relevant node does not have a local name (i.e. it’s a comment, root, or text node.)

string namespace-uri(node-set?)

Returns the namespace name of the first node in the argument node-set, or the namespace name of the context node if the argument is omitted. It returns an empty string if the node is an element or attribute that is not in a namespace. It also returns an empty string if namespace names don’t apply to this node (i.e. it’s a comment, processing instruction, root, or text node.)

string name(node-set?)

Returns the full, prefixed name of the first node in the argument node-set, or the name of the context node if the argument is omitted. It returns the empty string if the relevant node does not have a name (e.g. it’s a comment or text node.)

Boolean functions

boolean boolean(object)

Converts the argument to a boolean in a mostly sensible way. NaN and 0 are false. All other numbers are true. Empty strings are false. All other strings are true. Empty node-sets are false. All other node-sets are true.

boolean not(boolean)

This function turns true into false and false into true.

boolean true()

This function always returns true. It’s necessary because XPath does not have any boolean literals.

boolean false()

This function always returns false. It’s necessary because XPath does not have any boolean literals.

boolean lang(string)

This function returns true if the context node is written in the language specified by the argument. The language of the context node is determined by the currently in-scope xml:lang attribute. If there is no such attribute, this function returns false.

String functions

string string(object?)

This function returns the string-value of the argument. If the argument is a node-set, then it returns the string-value of the first node in the set. If the argument is omitted, it returns the string-value of the context node.

string concat(string, string, string...)

This function returns a string containing the concatenation of all its arguments.

boolean starts-with(string, string)

This function returns true if the first string starts with the second string. Otherwise it returns false.

boolean contains(string, string)

This function returns true if the first string contains the second string. Otherwise it returns false.

string substring-before(string, string)

This returns that part of the first string that precedes the second string. It returns the empty string if the second string is not a substring of the first string. If the second string appears multiple times in the first string, then this returns the portion of the first string before the first appearance of the second string.

string substring-after(string, string)

This returns that part of the first string that follows the second string. It returns the empty string if the second string is not a substring of the first string. If the second string appears multiple times in the first string, then this returns the portion of the first string after the initial appearance of the second string.

string substring(string, number, number?)

This returns the substring of the first argument beginning at the second argument and continuing for the number of characters specified by the third argument (or until the end of the string if the third argument is omitted.)

number string-length(string?)

Returns the number of Unicode characters in the string, or the string-value of the context node if the argument is omitted. This may not be the same as the number returned by the length() method in Java’s String class because XSLT counts characters and Java counts UTF-16 code points.

string normalize-space(string?)

This function strips all leading and trailing white-space from its argument, or the string-value of the context node if the argument is omitted, and condenses all other runs of whitespace to a single space. It’s very useful in XML documents where whitespace is used primarily for formatting.

string translate(string, string, string)

This function replaces all characters in the first string that are found in the second string with the corresponding character from the third string.

Number functions

number number(object?)

This function converts its argument to a number in a reasonable way. Strings like "23" and "42.5" are converted exactly as you’d expect. Other strings are converted to NaN. Node-sets are converted by converting the string-value of the first node in the set. True booleans are converted to 1. False booleans are converted to 0. If the argument is omitted, it converts the string-value of the context node to a number.

number sum(node-set)

Each node in the node-set is converted to a number, as if by the number() function. Those numbers are added together, and the sum is returned.

number floor(number)

Returns the largest integer less than or equal to the argument.

number ceiling(number)

Returns the smallest integer greater than or equal to the argument.

number round(number)

Returns the integer nearest to the argument.

There’s more to XPath than the basics I’ve covered here. In particular I haven’t discussed variables or extension functions, since both of these are normally only important when using XPath as part of XSLT or XQuery, rather than when using raw XPath in combination with Java. However, this should give you the basic knowledge you need to write simple XPath expressions and include those in your programs. Now it’s time to investigate the APIs that enable you to do this.

Prev | Up | Next
Location Paths | Home | XPath Engines
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 07, 2002


XPath Engines
Prev Chapter 16. XPath Next

XPath Engines

There are several good open source XPath engines for Java, most distributed as part of XSLT processors. These include:

Saxon 6.5.x

A very fast XSLT processor written by Michael Kay. It’s distributed under the Mozilla Public License 1.0. This is the processor I used to generate this book. (Saxon 7.x is also available, but it’s an incomplete experimental implementation of XPath 2.0, which itself likely won’t be finished until sometime in 2003. Both the Saxon 6.5 API and XPath 1.0 are much more stable and bug-free.)

Xalan-J

An XSLT processor used by several Apache XML projects including Cocoon. It is of course distributed under the Apache license. If you happen to work for one of those dinosaur companies with a firm policy against using free software you can buy the same product from IBM under the name LotusXSL.

Jaxen

A stand-alone XPath implementation that works with DOM, JDOM, dom4j, and ElectricXML.

Unfortunately, although the XPath data model and expression syntax are standardized, the API for integrating them into your Java programs is not. Each separate XPath engine does things differently. Saxon uses a custom DOM implementation and does not work with other DOM implementations like Xerces or Crimson. Xalan-J is also based on the DOM, but only requires a generic DOM. It isn’t limited to just the Apache XML Project’s Xerces DOM. Jaxen can work with any underlying data model, but the API still isn’t portable to other XPath engines. Other implementations do something different still. This means that your code tends to get fairly closely tied to the XPath engine you choose.

To demonstrate the different APIs, let’s revisit the Fibonacci SOAP client from Chapter 3. However, this time we’ll use XPath to extract just the parts we want. Recall that the body of each request document contains a calculateFibonacci element in the http://namespaces.cafeconleche.org/xmljava/ch3/ namespace. This element contains a single positive integer:

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >
  <SOAP-ENV:Body>
    <calculateFibonacci
      xmlns="http://namespaces.cafeconleche.org/xmljava/ch3/"
      type="xsi:positiveInteger">5</calculateFibonacci>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The server responds with a list of Fibonacci numbers enclosed in a SOAP response envelope. For example, here’s the response to a request for the first five Fibonacci numbers:

<?xml version="1.0"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" />
  <SOAP-ENV:Body>
    <Fibonacci_Numbers
      xmlns="http://namespaces.cafeconleche.org/xmljava/ch3/">
      <fibonacci index="1">1</fibonacci>
      <fibonacci index="2">1</fibonacci>
      <fibonacci index="3">2</fibonacci>
      <fibonacci index="4">3</fibonacci>
      <fibonacci index="5">5</fibonacci>
    </Fibonacci_Numbers>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The client needs to find all the fibonacci elements. There are many XPath expressions which will do this, the most obvious of which are:

//fibonacci
/SOAP-ENV:Envelope/SOAP-ENV:Body/Fibonacci_Numbers/fibonacci

However, there’s a catch. XPath expressions cannot match the default namespace. That is, the fibonacci element in the expression is in no namespace at all. It will not match fibonacci elements in the http://namespaces.cafeconleche.org/xmljava/ch3/ namespace. So instead you have to give it a prefix, even though it doesn’t have one in the original document. For example,

//f:fibonacci
/SOAP-ENV:Envelope/SOAP-ENV:Body/f:Fibonacci_Numbers/f:fibonacci

Having assigned it a prefix, you then have to map that prefix to a namespace URI. Indeed you have to do this for the SOAP-ENV prefix as well since the prefix will be used in a Java program instead of in the XML document where it was defined. Exactly how you do this varies from API to API, but generally you’ll pass some collection of namespace bindings as an argument to the method that evaluates the expression, as well as the expression itself.

The second of these two location paths is more efficient in general. The // operator, and indeed any location step that uses the descendant, descendant-or-self, ancestor, or ancestor-or-self axis is generally going to be slow relative to a more explicit spelling out of the hierarchy you expect. On the other hand, these axes are much more robust against unexpected changes in document structure. For example, //f:fibonacci would work even if somebody sent you an incorrect but well-formed document that left out the SOAP-ENV:Body element or used the SOAP 1.2 namespace instead of the SOAP 1.1 namespace. The more explicit path /SOAP-ENV:Envelope/SOAP-ENV:Body/f:Fibonacci_Numbers/f:fibonacci would not. Generally I recommend starting with the most robust path possible, and only using the more explicit paths if profiling proves performance is a problem. In the latter case, I would also seriously consider checking each document I received against a schema, and rejecting it immediately if it wasn’t valid.

XPath with Saxon

The Saxon 6.5 API is rather convoluted and involves over 200 different classes in 18 different packages. Fortunately you can ignore most of these for basic XPath searching. The most common sequence of steps to search a document is:

  1. Use JAXP to a build a Saxon Document object

  2. Attach the document to a Context object.

  3. Declare the namespaces used in the XPath expressions in a StandaloneContext.

  4. Make an Expression from the StandaloneContext and the string form of your XPath expression.

  5. Evaluate the Expression to return one of the four XPath data types.

Saxon requires a custom DOM that has been annotated with the information it needs. You can’t just pass in a Xerces DOM or a Crimson DOM. Thus before you use JAXP to parse the document you have to set the javax.xml.parsers.DocumentBuilderFactory system property to com.icl.saxon.om.DocumentBuilderFactoryImpl. Since you know this at compile time and do not want to allow the user to change it at runtime, use System.setProperty() in your code rather than passing it in on the command line. In case other parts of the program are using a different implementation, you should save the old value and restore it when you’re done. Otherwise, parsing a document with Saxon is just like using any other parser. For example,

  String oldFactory = System.getProperty(
   "javax.xml.parsers.DocumentBuilderFactory");
  System.setProperty("javax.xml.parsers.DocumentBuilderFactory",
   "com.icl.saxon.om.DocumentBuilderFactoryImpl");
   factory.setNamespaceAware(true);

  // Use the factory...

  if (oldFactory != null) {
    System.setProperty(
     "javax.xml.parsers.DocumentBuilderFactory", oldFactory);
  }

Once you’ve set the DocumentBuilderFactory, parse the input document as normal to produce a DOM Node or Document object. The exact type doesn’t really matter because you’ll immediately cast this to the Saxon implementation class com.icl.saxon.om.NodeInfo. For example,

  DocumentBuilder builder = factory.newDocumentBuilder();
  InputSource data = new InputSource(in); // InputSource is a SAX class
  Node doc = builder.parse(data); // Node is a DOM interface
  NodeInfo info = (NodeInfo) doc; // NodeInfo is a Saxon class

You’ll notice that Saxon freely mixes classes from SAX, DOM, TrAX, JAXP, and its internal implementation. A typical Saxon program imports a lot of packages.

Before this document can be searched, you’ll need to establish it as an XPath context node. Saxon uses the com.icl.saxon.Context class to represent context nodes. This is constructed with a no-args constructor. You then set its context node with the aptly named setContextNode() method like this:

  Context context = new Context();
  context.setContextNode(info);

Here the root node is the context node. However, you could use standard DOM methods to navigate through the tree and find another node to serve as the context node. Personally, I prefer to leave as much of the navigation work to XPath as possible.

The document we’ve just parsed defines its own namespace prefixes and URIs. However, these may not be the same ones used in the XPath expression. In particular, any default namespaces in the document will have to be mapped to prefixes in the XPath expression. As always in XPath, the namespaces matter. The prefixes don’t. There are two prefixes to map, SOAP-ENV and f. A Java program is not an XML document so these can’t be mapped in the customary way with xmlns attributes. Instead they have to be added to a Saxon com.icl.saxon.expr.StandaloneContext object. Each such object needs access to the document’s com.icl.saxon.om.NamePool to which the necessary namespaces can be added. This is all set up thusly:

  DocumentInfo docInfo = info.getDocumentRoot();
  NamePool pool = docInfo.getNamePool();
  StandaloneContext sc = new StandaloneContext(pool);
  sc.declareNamespace("SOAP-ENV",
   "http://schemas.xmlsoap.org/soap/envelope/");
  sc.declareNamespace("f",
   "http://namespaces.cafeconleche.org/xmljava/ch3/");

That does it for the preliminaries. We’re finally ready to search the document with XPath. The Saxon class that both represents and evaluates XPath expressions is com.icl.saxon.expr.Expression. You pass a String containing the XPath expression and the StandaloneContext object to the static Expression.make() factory method. This returns an Expression object. You then pass the Context object and a boolean specifying whether you want the result to be sorted in document order to the enumerate() method. This returns a com.icl.saxon.om.NodeEnumeration, one of Saxon’s representations of node-sets. For example,

Expression xpath = Expression.make(
 "/SOAP-ENV:Envelope/SOAP-ENV:Body/f:Fibonacci_Numbers/f:fibonacci",
 sc);
NodeEnumeration enum = xpath.enumerate(context, true);
while (enum.hasMoreElements()) {
  NodeInfo result = enum.nextElement();
  System.out.println(result.getStringValue());
}

The NodeEnumeration class is modeled after (but does not extend) the Enumeration interface in the java.util package. It allows you to iterate through the returned node-set. Each node in this set implements the Saxon NodeInfo interface. The getStringValue() method in this interface returns the XPath string-value of that node.

NodeEnumeration is limited to a single use. That is, you cannot set it back to its beginning and iterate through a second time. If you need a persistent result, you can call evaluateAsNodeSet() which returns a com.icl.saxon.expr.NodeSetValue instead. You can then sort and enumerate this object repeatedly. For example,

Expression xpath = Expression.make(
 "/SOAP-ENV:Envelope/SOAP-ENV:Body/f:Fibonacci_Numbers/f:fibonacci",
 sc);
NodeSetValue set = xpath.evaluateAsNodeSet(context);
set.sort();
NodeEnumeration enum = set.enumerate();
while (enum.hasMoreElements()) {
  NodeInfo result = enum.nextElement();
  System.out.println(result.getStringValue());
}

Alternately, if you want the expression to return a number, string, or boolean, you can call one of these three methods instead:

public boolean evaluateAsBoolean(Context context)
    throws XPathException;
public double evaluateAsNumber(Context context)
    throws XPathException;
public String evaluateAsString(Context context)
    throws XPathException;

If the expression returns the wrong type, then Saxon will convert the result as if by the XPath number(), string(), or boolean() function. The only conversion Saxon can’t perform is a primitive type to a node-set. If you try to evaluate an expression that returns one of the three basic types as a node-set, then evaluateAsNodeSet() throws an XPathException.

I can now show you the complete method that takes as an argument the InputStream from which the response document will be read and searches out the relevant parts with XPath:

  public static void readResponse(InputStream in)
   throws IOException, SAXException,  XPathException,
   ParserConfigurationException, TransformerException {

    String oldFactory = System.getProperty(
     "javax.xml.parsers.DocumentBuilderFactory");
    System.setProperty(
     "javax.xml.parsers.DocumentBuilderFactory",
     "com.icl.saxon.om.DocumentBuilderFactoryImpl");
    DocumentBuilderFactory factory
     = DocumentBuilderFactory.newInstance();
    factory.setNamespaceAware(true);

    DocumentBuilder builder = factory.newDocumentBuilder();

    InputSource data = new InputSource(in);
    Node doc = builder.parse(data);
    NodeInfo info = (NodeInfo) doc;
    Context context = new Context();
    context.setContextNode(info);

    NamePool pool = info.getDocumentRoot().getNamePool();
    StandaloneContext sc = new StandaloneContext(pool);
    sc.declareNamespace("SOAP",
     "http://schemas.xmlsoap.org/soap/envelope/");
    sc.declareNamespace("f",
     "http://namespaces.cafeconleche.org/xmljava/ch3/");

    Expression xpath = Expression.make(
     "/SOAP:Envelope/SOAP:Body/f:Fibonacci_Numbers/f:fibonacci",
     sc);
    NodeEnumeration enum = xpath.enumerate(context, true);
    while (enum.hasMoreElements()) {
      NodeInfo result = enum.nextElement();
      System.out.println(result.getStringValue());
    }

    // Restore the original factory
    if (oldFactory != null) {
      System.setProperty(
       "javax.xml.parsers.DocumentBuilderFactory", oldFactory);
    }

  }

Honestly, this is a little convoluted and perhaps more complex than the pure DOM, JDOM, or SAX equivalent. However, the advantage is that the code is never more complex than this. As the documents you’re searching grow in complexity, the XPath expressions become only slightly more complex and the Java code becomes no more complex than what you see here. The more details you can defer to the declarative XPath syntax the simpler and more robust your program will be.

XPath with Xalan

The Xalan-J XSLT processor from the Apache XML Project also includes an XPath API that’s useful for navigation in DOM programs. Underneath the hood, the basic design is strikingly similar to Saxon’s for two independently developed programs. However, Xalan does have one class that Saxon doesn’t which significantly simplifies life for developers, org.apache.xpath.XPathAPI. This class, shown in Example 16.5 provides static methods that handle many simple use-cases without lots of preliminary configuration.

Example 16.5. The Xalan XPathAPI class

package org.apache.xpath;

public class XPathAPI {

  public static Node selectSingleNode(Node context, String xpath)
   throws TransformerException;
  public static Node selectSingleNode(Node context, String xpath,
   Node namespaceContextNode) throws TransformerException;
  public static NodeIterator selectNodeIterator(Node context,
   String xpath) throws TransformerException;
  public static NodeIterator selectNodeIterator(Node context,
   String xpath, Node namespaceContextNode)
   throws TransformerException;
  public static NodeList selectNodeList(Node context,
   String xpath) throws TransformerException;
  public static NodeList selectNodeList(Node context,
   String xpath, Node namespaceContextNode)
   throws TransformerException;
  public static XObject eval(Node context, String xpath)
   throws TransformerException;
  public static XObject eval(Node context, String xpath,
   Node namespaceContextNode) throws TransformerException;
  public static XObject eval(Node context, String xpath,
   PrefixResolver prefixResolver) throws TransformerException;

}

Each method in this class takes two or three arguments:

  • The context node as a DOM Node object

  • The XPath expression as a String

  • The namespace prefix mappings as a DOM Node object or a Xalan PrefixResolver object. This can be omitted if the XPath expression does not use any namespace prefixes.

The methods differ primarily in return type. There are four possible return types:

  • A single DOM Node

  • A DOM NodeList

  • A DOM traversal NodeIterator

  • A Xalan XObject

The first three types you’ve encountered in previous chapters. I won’t say anything more about them here except not to use the methods that only return a single Node. They are fragile against unexpected changes in document format. Trust me on this one. No matter how sure you are that all the documents you’re processing contain exactly one node that matches an XPath expression, sooner or later you’re going to encounter a document that is either missing the node completely or has two or more. List iteration is far more reliable than selecting a single node. If you disregard this warning and use selectSingleNode() anyway, then by all means use a schema to validate your document before accepting it for processing.

The XObject type is new. This is a class in the org.apache.xpath.objects package that represents the various kinds of XPath objects —string, number, boolean, and node-set—as well as a few XSLT objects—result tree fragment, unknown types, and unresolved variables. This class has a number of methods, mostly intended for use in XSLT. For XPath, all you really need are these five methods for converting an XObject to a more specific type:

public boolean bool()
    throws TransformerException;
public double num()
    throws TransformerException;
public String str()
    throws TransformerException;
public NodeIterator nodeset()
    throws TransformerException;
public NodeList nodelist()
    throws TransformerException;

The last two methods only work if the XObject returned by the eval() method is in fact a node-set. Otherwise, they throw a TransformerException.

For the sake of comparison, let’s look at how we’d use these classes to solve the Fibonacci SOAP client problem addressed earlier by Saxon.

  public static void readResponse(InputStream in)
   throws IOException, SAXException, TransformerException,
   ParserConfigurationException {

    DocumentBuilderFactory factory
     = DocumentBuilderFactory.newInstance();
    factory.setNamespaceAware(true);
    DocumentBuilder builder = factory.newDocumentBuilder();

    InputSource data = new InputSource(in);
    Node doc = builder.parse(data);

    // set up a document purely to hold the namespace mappings
    DOMImplementation impl = builder.getDOMImplementation();
    Document namespaceHolder = impl.createDocument(
     "http://namespaces.cafeconleche.org/xmljava/ch3/",
     "f:namespaceMapping", null);
    Element root = namespaceHolder.getDocumentElement();
    root.setAttributeNS("http://www.w3.org/2000/xmlns/", "xmlns:SOAP",
     "http://schemas.xmlsoap.org/soap/envelope/");
    root.setAttributeNS("http://www.w3.org/2000/xmlns/", "xmlns:f",
     "http://namespaces.cafeconleche.org/xmljava/ch3/");

    NodeList results = XPathAPI.selectNodeList(doc,
     "/SOAP:Envelope/SOAP:Body/f:Fibonacci_Numbers/f:fibonacci",
     root);
    for (int i = 0; i < results.getLength(); i++) {
      Node result = results.item(i);
      XObject value = XPathAPI.eval(result, "string()");
      System.out.println(value.str());
    }

  }

The input document is parsed in the usual way with the JAXP DocumentBuilder class. JAXP is also used to create a new Element that provides namespace bindings for the XPath expression. Alternately, I could have implemented the org.apache.xml.utils.PrefixResolver interface in a separate class and used that instead, but using a node is simpler.

The XPath expression, the input document, and the context node (the root element here) are passed to XPathAPI.selectNodeList() to find all the matching elements. This returns a standard DOM NodeList which can be iterated through in the usual way. The last XPath axis in the expression, child, is a forward axis so this list is sorted in document order. The string-value of each node in this list is determined by calling the XPath string() function with the node as the context. This returns an instance of the Xalan class XObject which can be converted to a Java String using the str() method. The final result is printed on System.out.

One crucial difference you’ll note between Xalan and Saxon is that at no point does Xalan require or use any specific classes from the DOM implementation. All the DOM nodes are generic DOM nodes. Thus in theory this same code should work with any DOM2 and JAXP 1.1 compliant implementation. In practice, I’ve verified that it does work with Xerces and Crimson but not with GNU JAXP.

Prev | Up | Next
Expressions | Home | DOM Level 3 XPath
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified March 06, 2002


DOM Level 3 XPath
Prev Chapter 16. XPath Next

DOM Level 3 XPath

The Saxon API only works with Saxon. The Xalan API only works with Xalan. Both only work with Java. The W3C DOM Working Group is attempting to define a standard, cross-engine XPath API that can be used with many different XPath engines (though as of Summer 2002 this effort is just beginning and is not yet supported by any implementations). DOM Level 3 includes an optional XPath module in the org.w3c.dom.xpath package. The feature string "XPath" with the version "3.0" tests for the presence of this module. For example,

if (!impl.hasFeature("XPath", "3.0")) {
  System.err.println("This DOM implementation does not support XPath");
  return;
}

Caution

This section is based on the March 28, 2002, working draft of the Document Object Model (DOM) Level 3 XPath Specification. The details are still subject to change, however.

The XPath module has two main interfaces, XPathEvaluator and XPathResult. XPathEvaluator, shown in Example 16.6 searches an XML document for the objects identified by an XPath expression such as /book/chapter/section[starts-with(@title, 'DOM')]. The XPath expression is passed as a Java String, the context node as a DOM Node object. The result of evaluating the expression is returned as an XPathResult, a wrapper interface for the four standard XPath data types: node-set, string, boolean, and number.

Example 16.6. The XPathEvaluator interface

package org.w3c.dom.xpath;

public interface XPathEvaluator {

  public XPathResult evaluate(String expression,
   Node contextNode, XPathNSResolver resolver, short type,
   XPathResult result) throws XPathException, DOMException;

  public XPathExpression createExpression(String expression,
   XPathNSResolver resolver) throws XPathException, DOMException;
  public XPathNSResolver createNSResolver(Node nodeResolver);

}

In DOM implementations that support XPath, the same classes that implement org.w3c.dom.Document implement XPathEvaluator. Thus no special constructor or factory class is required. Just cast the Document object that encapsulates the document you want to query to XPathEvaluator (after checking to make sure that the implementation supports XPath with hasFeature(), of course). For example, in Chapter 5 you saw an XML-RPC server that returned Fibonacci numbers. The documents that server returned looked like this:

<?xml version="1.0"?>
<methodResponse>
  <params>
    <param>
      <value><double>28657</double></value>
    </param>
  </params>
</methodResponse>

A client for this program needs to extract the content of the double element. You can use XPath to simplify this task. There are numerous XPath expressions that will retrieve the relevant node. These include:

  • /methodResponse/params/param/value/double

  • /child::methodResponse/child::params/child::param/child::value/child::double[1]

  • /methodResponse/params/param/value/double[1]

  • //double[1]

  • /descendant::double[1]

Depending on what you intend to do with the node once you have it, you might want to use one of the functions that returns the string-value of the node instead. In that case, these expressions would be appropriate:

  • normalize-space(/methodResponse/params/param/value/double)

  • normalize-space(//double[1])

  • string(//double)

  • normalize-space(/methodResponse)

  • normalize-space(/)

These are all absolute expressions that do not depend on the context node. There are many more depending on what the context node is. For example, if the context node were set to the methodResponse document element, then these relative location paths and function calls would also work:

  • params/param/value/double

  • child::params/child::param/child::value/child::double[1]

  • .//double

  • normalize-space(.//double[1])

  • normalize-space(params)

  • normalize-space(/)

Assuming the relevant server response has already been parsed into a DOM Document object named response, the following code will extract the desired element into an XPathResult object:

Document response;
// Initialize response object by parsing request...
String query = "/methodResponse/params/param/value/double";
if (impl.hasFeature("XPath", "3.0")) {
  XPathEvaluator evaluator = (XPathEvaluator) response;
  try {
    XPathResult index = evaluator.evaluate(query, response,
     null, XPathResult.ORDERED_NODE_ITERATOR_TYPE, null)
    // work with the result...
  }
  catch (XPathException e) {
    System.err.println(query
     + " is not a correct XPath expression");
  }
  catch (DOMException e) {
    System.err.println(e);
  }
}

What this builds is an XPathResult object, which is one step removed from the string you actually want. The XPathResult interface is a wrapper for the four things an XPath expression might evaluate to (double, string, boolean, or node-set). Getter methods are provided to return the relevant type from the XPathResult. Example 16.7 shows this interface.

Example 16.7. The XPathResult interface

package org.w3c.dom.xpath;

public interface XPathResult {

  public static final short ANY_TYPE                     = 0;
  public static final short NUMBER_TYPE                  = 1;
  public static final short STRING_TYPE                  = 2;
  public static final short BOOLEAN_TYPE                 = 3;
  public static final short UNORDERED_NODE_ITERATOR_TYPE = 4;
  public static final short ORDERED_NODE_ITERATOR_TYPE   = 5;
  public static final short UNORDERED_NODE_SNAPSHOT_TYPE = 6;
  public static final short ORDERED_NODE_SNAPSHOT_TYPE   = 7;
  public static final short ANY_UNORDERED_NODE_TYPE      = 8;
  public static final short FIRST_ORDERED_NODE_TYPE      = 9;

  public short   getResultType();

  public double  getNumberValue() throws XPathException;
  public String  getStringValue() throws XPathException;
  public boolean getBooleanValue() throws XPathException;
  public Node    getSingleNodeValue() throws XPathException;

  public boolean getInvalidIteratorState();
  public int     getSnapshotLength() throws XPathException;
  public Node    iterateNext() throws XPathException, DOMException;
  public Node    snapshotItem(int index) throws XPathException;

}

Of the four getXXXValue() methods, only one of them will actually return a sensible result for any given XPath expression. The other three will throw an XPathException with the error code XPathException.TYPE_ERR. The above example expected only a single node as a result of evaluating the XPath location path /methodResponse/params/param/value/double. Consequently, the getSingleNodeValue() method can retrieve it:

Element doubleNode = (Element) index.getSingleNodeValue();

That this expression returns a single value is indicated by foreknowledge of the input format, not by anything intrinsic to the XPath expression. If there was more than one double element in the client request, then the location path would find them all.

Now we have an Element node, but what we really need is the complete text of that node, after accounting for possible if unlikely comments, CDATA sections, processing instructions, and other detritus that DOM presents us with. In Chapter 10 I developed a getFullText() utility method to account for this, and I could use it again here. However, DOM XPath offers a simpler solution. The getStringValue() method returns the XPath value of the node-set. The XPath value of an element node is defined as the complete text of the node after all character references, entity references, and CDATA sections are resolved and all other markup is stripped. Thus instead of requesting a Node, you can ask for a String:

String value = index.getStringValue();

Or maybe it’s not a String you want but a number. In this case, use getNumberValue() which returns a double:

double value = index.getNumberValue();

The DOM 3 XPath methods getStringValue(), getNumberValue(), and getBooleanValue() correspond to the XPath casting functions string(), number(), and boolean(). XPath has a number of other useful functions. For instance, normalize-space() first converts its argument to a string as if by the string() function and then strips all leading and trailing white space and converts all other runs of whitespace to a single space. Using this function, you can use a simpler location path:

  XPathResult index = evaluator.evaluate("normalize-space(/)",
   response, null, XPathResult.ORDERED_NODE_ITERATOR_TYPE, null)
  String value = index.getStringValue();

In the case of an XPath expression that evaluates to a node-set, getSingleNodeValue() only returns the first node in the set. You can invoke iterateNext() and then call getSingleNodeValue() again to get the second node in the set. Repeat this procedure for the third node, the fourth node, and so on until getSingleNodeValue() returns null, indicating that there are no more nodes in the set. If the set is empty to begin with, then getSingleNodeValue() returns null immediately. This is how you’d handle a case like the SOAP response that returns multiple int elements.

Namespace Bindings

Because the SOAP request document uses namespace qualified elements, however, we’ll first have to provide some namespace bindings that can be used when evaluating the XPath expression. The XPathNSResolver interface provides the namespace bindings. Although you can implement this in any convenient class, an instance is normally created by passing a Node with all necessary bindings to the createNSResolver() method of the XPathEvaluator interface. For example, this code uses JAXP to build a very simple document whose document element binds the prefix SOAP to the URI http://schemas.xmlsoap.org/soap/envelope/ and the prefix f to the URI http://namespaces.cafeconleche.org/xmljava/ch3/. Then that document element is passed to the XPathEvaluator’s createNSResolver() method to create an XPathNSResolver object that has the same namespace bindings as the synthetic node we created.

   // Load JAXP
   DocumentBuilderFactory factory
    = DocumentBuilderFactory.newInstance();
   factory.setNamespaceAware(true);
   DocumentBuilder builder = factory.newDocumentBuilder();

   // Build the document
   DOMImplementation impl = builder.getDOMImplementation();
   Document namespaceHolder = impl.createDocument(
    "http://namespaces.cafeconleche.org/xmljava/ch3/",
    "f:namespaceMapping", null);

   // Attach the namespace declaration attributes
   Element root = namespaceHolder.getDocumentElement();
   root.setAttributeNS("http://www.w3.org/2000/xmlns/",
    "xmlns:SOAP", "http://schemas.xmlsoap.org/soap/envelope/");
   root.setAttributeNS("http://www.w3.org/2000/xmlns/",
    "xmlns:f", "http://namespaces.cafeconleche.org/xmljava/ch3/");

   // Create the resolver
   XPathNSResolver namespaces = evaluator.createNSResolver(root);

Now we’re ready to repeat the earlier example, but this time using the DOM XPath API instead of the processor specific Xalan or Saxon APIs. To relieve the tedium, I’m going to make a small shift in the pattern of the readResponse() method. Instead of storing the XPath search string and the namespace bindings in the source code, I’m going to split them out into the separate XML document shown in Example 16.8 that can be bundled with the application and is assumed to live at the relative URL config.xml. (A more realistic example could store this document as a resource in the application’s JAR archive.)

Example 16.8. An XML document containing namespace bindings and an XPath search expression

<?xml version="1.0"?>
<config
search="/SOAP:Envelope/SOAP:Body/f:Fibonacci_Numbers/f:fibonacci"
 xmlns:f="http://namespaces.cafeconleche.org/xmljava/ch3/"
 xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/"
/>

The program both reads the XPath expression from search attribute of the document element and uses that element for the namespace bindings. This enables the XPath string to change independently of the source code.

Here’s the configurable, DOM-XPath based readResponse() method. Since the iterator always returns a DOM node, we have to use a second XPath evaluation on each node to take the element node’s string value.

  public static void readResponse(InputStream in)
   throws IOException, SAXException, DOMException,
   XPathException, ParserConfigurationException {

    DocumentBuilderFactory factory
     = DocumentBuilderFactory.newInstance();
    factory.setNamespaceAware(true);
    DocumentBuilder builder = factory.newDocumentBuilder();

    // Parse the server response
    InputSource data = new InputSource(in);
    Node doc = builder.parse(data);

    // Check to see that XPath is supported
    if (!impl.hasFeature("XPath", "3.0")) {
      throw new XPathException(
       "Implementation does not support XPath");
    }
    XPathEvaluator evaluator = (XPathEvaluator) doc;

    // Parse the config file
    Document config = builder.parse("config.xml");
    Element root    = config.getDocumentElement();
    String query    = root.getAttributeValue("search");
    XPathNSResolver namespaces
     = evaluator.createNSResolver(root);

    // Evaluate the expression
    XPathResult nodes = evaluator.evaluate(
     "/SOAP:Envelope/SOAP:Body/f:Fibonacci_Numbers/f:fibonacci",
     doc, namespaces, XPathResult.ORDERED_NODE_ITERATOR_TYPE,
     null);
    // work with the result...

    Node next;
    while (next = nodes.iterateNext()) {
      XPathResult stringValue = evaluator.evaluate("string()",
       next, namespaces, XPathResult.STRING_TYPE, null);
      System.out.println(stringValue.getStringValue());
    }

  }

Snapshots

Iterators like this one are only good for a single pass. You cannot reuse them or back up in them. Furthermore, if the Document object over which the iterator is traversing changes before you're finished with the iterator (e.g. a node in the iterator is deleted from the Document object) then iterateNext() throws a DOMException with the code INVALID_STATE_ERR.

To hold on to a more stable list that can be reused and survives document edits, request a snapshot of the node-set returned rather than an iterator. A snapshot is reusable and features random access through indexing. For example, using a snapshot the above code would become:

    // Evaluate the expression
    XPathResult nodes = evaluator.evaluate(
     "/SOAP:Envelope/SOAP:Body/f:Fibonacci_Numbers/f:fibonacci",
     doc, namespaces, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE,
     null);

    for (int i = 0; i < nodes.getSnapshotLength(); i++) {
      Node next = nodes.snapshotItem(i);
      XPathResult stringValue = evaluator.evaluate("string()",
       next, namespaces, XPathResult.STRING_TYPE, null);
      System.out.println(stringValue.getStringValue());
    }

  }

Of course, snapshots have the opposite problem: there is no guarantee that the nodes in the snapshot reflect the current state of the Document.

Compiled Expressions

An XPath engine that implements the DOM XPath API may need to compile the expression into some internal form rather than simply keeping it as a generic String. The XPathExpression interface, shown in Example 16.9, represents such a compiled expression.

Example 16.9. The DOM3 XPathExpression interface

package org.w3c.dom.xpath;

public interface XPathExpression {

  public XPathResult evaluate(Node contextNode, short type,
   XPathResult result) throws XPathException, DOMException;

}

You can use the createExpression() method in the XPathEvaluator interface to compile a String into an XPathExpression:

public XPathExpression createExpression(String expression, XPathNSResolver resolver)
    throws XPathException;

Then you can repeatedly invoke the same expression on different documents without having to compile the XPath expression from a string each time. For example,

  XPathExpression expression = evaluator.createExpression(
   "/SOAP:Envelope/SOAP:Body/f:Fibonacci_Numbers/f:fibonacci",
   namespaces);
  XPathResult nodes = expression.evaluate(doc,
  XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null);

This isn’t very important for an expression that’s only going to be used once or twice. However, in a program that will process many documents in the same way, it can be a significant savings. Imagine, for example, an XML-RPC or SOAP server that receives thousands of requests an hour and needs to apply the same XPath expression to each request document. The exact amount of speed you’ll gain by compiling your expressions will of course vary from implementation to implementation.

Prev | Up | Next
XPath Engines | Home | Jaxen
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 04, 2002


Jaxen
Prev Chapter 16. XPath Next

Jaxen

The Jaxen Java XPath Engine is an open source cross-API (DOM, JDOM, dom4j, and ElectricXML) XPath library for Java. Whereas DOM3 XPath attempts to be a cross-implementation, cross-language XPath API for the Document Object Model alone, Jaxen attempts to be a cross-model API for its own XPath engine. It is a Java class library that can operate on various XML object models using a standard engine rather than an API that can be offered by many different engines for one model. It allows you to pass DOM, JDOM, dom4j, and ElectricXML objects directly to XPath functions like position() and translate(). Furthermore, whereas DOM 3 XPath offers fairly rudimentary interfaces for evaluating XPath expressions in a a particular document against a context node, Jaxen is a more complete object model for XPath expressions.

Jaxen’s class library is quite large, on a par with Saxon’s or Xalan’s. It includes classes representing each of XPath’s functions, iterators, node tests, and axes. Fortunately, you don’t need to know all that to perform simple searches. Indeed, Jaxen’s basic API is probably the simplest of the XPath APIs discussed in this chapter. The main interface you need is XPath. There are four implementations of this interface in four different packages, one each for DOM, JDOM, dom4j, and ElectricXML: org.jaxen.dom.DOMXPath, org.jaxen.jdom.JDOMXPath, org.jaxen.dom4j.Dom4jXPath, and org.jaxen.exml.ElectricXPath. I’ll demonstrate this API with the DOM implementation, but the patterns are much the same for the other three APIs.

The following steps search an XML document with Jaxen:

  1. Construct an XPath object by passing a String containing an XPath expression to the model-specific constructor:

    public DOMXPath(String expression)
        throws JaxenException;
    public JDOMXPath(String expression)
        throws JaxenException;
    public Dom4jXPath(String expression)
        throws JaxenException;
    public ElectricXPath(String expression)
        throws JaxenException;

  2. Set the namespace bindings by calling addNamespace() for each namespace binding the XPath expression uses:

    public void addNamespace(String prefix, String uri)
        throws JaxenException;

    (You can skip this step if the XPath expression doesn’t use any prefixed names.)

  3. Invoke one of the following methods to evaluate the expression, depending on what type of result you expect or want:

    public Object evaluate(Object context)
        throws JaxenException;
    public List selectNodes(Object context)
        throws JaxenException;
    public Object selectSingleNode(Object context)
        throws JaxenException;
    public String stringValueOf(Object context)
        throws JaxenException;
    public boolean booleanValueOf(Object context)
        throws JaxenException;
    public Number numberValueOf(Object context)
        throws JaxenException;

For an example, let’s rewrite the Fibonacci XML-RPC client one last time. However, this time, I’ll use Jaxen:

  public static void readResponse(InputStream in)
   throws IOException, SAXException, TransformerException,
   ParserConfigurationException, JaxenException {

    DocumentBuilderFactory factory
     = DocumentBuilderFactory.newInstance();
    factory.setNamespaceAware(true);
    DocumentBuilder builder = factory.newDocumentBuilder();

    InputSource data = new InputSource(in);
    Node doc = builder.parse(data);

    // There are different XPath classes in different packages
    // for the different APIs Jaxen supports
    XPath expression = new org.jaxen.dom.DOMXPath(
     "/SOAP:Envelope/SOAP:Body/f:Fibonacci_Numbers/f:fibonacci");
    expression.addNamespace("f",
     "http://namespaces.cafeconleche.org/xmljava/ch3/");
    expression.addNamespace("SOAP",
     "http://schemas.xmlsoap.org/soap/envelope/");
    Navigator navigator = expression.getNavigator();

    List results = expression.selectNodes(doc);
    Iterator iterator = results.iterator();
    while (iterator.hasNext()) {
      Node result = (Node) iterator.next();
      String value = StringFunction.evaluate(result, navigator);
      System.out.println(value);
    }

  }

As usual, first JAXP reads the document from the InputStream. Next a new Jaxen DOMXPath object is constructed from the String form of the XPath expression. If I were using Jaxen on top of JDOM, I would have constructed an org.jaxen.jdom.JDOMXPath object here instead. If I were using Jaxen on top of dom4j, I would have constructed an org.jaxen.dom4j.Dom4jXPath object here instead.

After the expression is created, I immediately bind all the namespace prefixes it uses by invoking the addNamespace() method. This location path uses two different namespace prefixes, so I call addNamespace() twice. Then I get the expression’s Navigator by invoking getNavigator(). In more advanced programs, you can use the Navigator class directly to move around the tree. Here, however, I just need this to pass as an argument to another method in a few lines. Finally, I pass the context node to the selectNodes() method to get a list of all the nodes in that document that satisfy the location path. In this case, the context node is the document itself because the location path is an absolute path. However, in other programs you might well pass an element or some other kind of node instead.

Jaxen’s selectNodes() method returns a standard java.util.List which can be iterated through in the usual way. Since this XPath expression operated on a DOM document and returned a node-set, the items in the list are all some form of DOM Node objects. This location path only selected elements so here they’re actually all DOM Element objects. If the Jaxen XPath were operating on a JDOM document, then the list would contain JDOM objects. If the Jaxen XPath were operating on a dom4j document, then the list would contain dom4j objects, and so on. In most cases you’ll want to cast the item in the list to some more specific type before continuing.

As the program iterates through the list, it deals with each node independently. I could use the DOM methods discussed in Chapters 9 through 13 to work with these nodes. However, what I really want is to get the XPath string-value of each node. This is provided by the Xpath string() which Jaxen represents as the org.jaxen.function.StringFunction class. The evaluate() method in this class applies the XPath string function to a specified object (here a DOM Node) in the context of a particular Jaxen Navigator. It returns the XPath string-value of the object.

Note

This is a little more convoluted than it perhaps needs to be because there’s no XPath 1.0 way to return a list of strings. For example, string(node-set) returns the string-value of the first node in the set rather than a list of the string-values of each node in the set. That’s why I have to move from XPath to DOM (where I can work with lists and sets) and back to XPath again rather than working with a single XPath expression that returns the final result. XPath is not Turing complete. Some of the logic is going to have to be implemented in Java.

In fact, Jaxen’s org.jaxen.function package provides Java representations of most of the functions in XPath 1.0: BooleanFunction, CeilingFunction, ConcatFunction, etc. Each of these classes has a static evaluate() method that invokes the function and returns the result. The argument lists and return types of this method change from function to function as appropriate for each function. In a few cases where the XPath function has a variable length argument list, the Jaxen function class uses overloaded evaluate() methods instead. These classes and their corresponding evaluate() methods are:

BooleanFunction
public static Boolean evaluate(Object o, Navigator navigator);
CeilingFunction
public static Double evaluate(Object o, Navigator navigator);
ConcatFunction
public static String evaluate(List list, Navigator navigator);
ContainsFunction
public static Boolean evaluate(Object string, Object match, Navigator navigator);
CountFunction
public static Number evaluate(Object node-set);
FalseFunction
public static Boolean evaluate();
FloorFunction
public static Double evaluate(Object o, Navigator navigator);
IdFunction
public static List evaluate(List contextNodes, Object arg, Navigator navigator);
LastFunction
public static Double evaluate(Context context);
LocalNameFunction
public static String evaluate(List node-set, Navigator navigator);
NameFunction
public static String evaluate(List node-set, Navigator navigator);
NamespaceUriFunction
public static String evaluate(List node-set, Navigator navigator);
NormalizeSpaceFunction
public static String evaluate(Object string, Navigator navigator);
NotFunction
public static Boolean evaluate(Object object, Navigator navigator);
NumberFunction
public static Double evaluate(Object object, Navigator navigator);
PositionFunction
public static Double evaluate(Context context);
RoundFunction
public static Double evaluate(Object object, Navigator navigator);
StartsWithFunction
public static Boolean evaluate(Object string, Object match, Navigator navigator);
StringFunction
public static String evaluate(Object object, Navigator navigator);
StringLengthFunction
public static Number evaluate(Object object, Navigator navigator);
SubstringAfterFunction
public static String evaluate(Object string, Object match, Navigator navigator);
SubstringBeforeFunction
public static String evaluate(Object string, Object match, Navigator navigator);
SubstringFunction
public static String evaluate(Object string, Object start, Navigator navigator);
public static String evaluate(Object string, Object start, Object length, Navigator navigator);
SumFunction
public static Double evaluate(Object node-set, Navigator navigator);
TranslateFunction
public static Boolean evaluate(Object original, Object from, Object to, Navigator navigator);
TrueFunction
public static Boolean evaluate();
Prev | Up | Next
DOM Level 3 XPath | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 04, 2002


Summary
Prev Chapter 16. XPath Next

Summary

XPath is a straightforward declarative language for selecting particular subsets of nodes from an XML document. Its data model is not quite the same as DOM’s, but that’s normally not a major problem. In fact in some cases such as taking the string-value of an element, the XPath data model is likely to be a lot closer to what you want than the DOM data model.

XPath location paths are composed of one or more location steps. Each location step has an axis and a node test, and may have one or more predicates. Each location step is evaluated with respect to the context nodes determined by the previous step in the path. The axis determines in which direction you move from the context node. The node test determines which nodes are selected along that axis, and the predicate decides which of the selected nodes are retained in the set.

A location path is actually just one kind of the more generic XPath expressions. Besides node-sets, XPath expressions can return doubles, strings, and booleans, which are pretty much the same as the Java types of the same name, with a few minor differences you normally don’t have to worry about. XPath offers the usual arithmetic and relational operators for working with these data types, as well as a library of more than two dozen useful functions.

Most XSLT processors have APIs that allow you to search XML documents with XPath expressions. The two most popular are Saxon and Xalan. Saxon’s API requires a custom DOM, whereas Xalan can work with pretty much any complete and correct DOM implementation.

DOM Level 3 XPath is a developing standard for using XPath in DOM programs that can be implemented across different processors, though as yet it isn’t implemented by any. It provides a reasonably simple API for saying, “Here’s a document, a context node, and a location path. Find me all the nodes from the document that match.”

Jaxen is a somewhat more ambitious cross-model effort that attempts to model XPath expressions themselves rather than just treating them as opaque strings. For example, Jaxen provides Java classes that represent all the different XPath functions so that you can pass Java objects like a Node or an Element directly to XPath functions like normalize-space() or namespace-uri(). More importantly, Jaxen works across different XML object models including not just DOM but also JDOM, dom4j, and ElectricXML.

Prev | Up | Next
Jaxen | Home | Chapter 17. XSLT
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 02, 2002


Chapter 17. XSLT
Prev Next

Chapter 17. XSLT

Table of Contents

XSL Transformations
Template Rules
Stylesheets
Taking the Value of a Node
Applying Templates
The Default Template Rules
Selection
Calling Templates by Name
TrAX
Thread Safety
Locating Transformers
The xml-stylesheet processing instruction
Features
XSLT Processor Attributes
URI Resolution
Error Handling
Passing Parameters to Style Sheets
Output Properties
Sources and Results
Extending XSLT with Java
Extension Functions
Extension Elements
Summary

Extensible Stylesheet Language Transformations (XSLT) is provably Turing-complete. That is, given enough memory an XSLT stylesheet can perform any calculation a program written in any other language can perform. However, XSLT is not designed as a general purpose programming language; and attempting to use it as one inevitably causes pain (especially if you’re accustomed to procedural languages like Java instead of functional languages like Scheme). Instead XSLT is designed as a templating language. Used in this manner, it is extremely flexible, powerful, and easy to use. However, you do have to recognize what it is and is not good for. You could calculate Fibonacci numbers in XSLT, but Java will do a much better job of that. You could write a Java program that converts DocBook documents to XHTML, but XSLT will make the task much easier. Use the right tool for the right job. Fortunately, XSLT style sheets can be combined with Java programs so that each tool can be applied to the parts of the job for which it’s appropriate.

XSL Transformations

XSLT is a transformation language. An XSLT stylesheet describes how documents in one format are converted to documents in another format. Both input and output documents are represented by the XPath data model. XPath expressions select nodes from the input document for further processing. Templates containing XSLT instructions are applied to the selected nodes to generate new nodes that are added to the output document. The final result document can be identical to the input document, a little different, or a lot different. Most of the time it’s somewhere in the middle.

XSLT is based on the notion of templates. An XSLT stylesheet contains semi-independent templates for each element or other node that will be processed. An XSLT processor parses the stylesheet and an input document. Then it compares the nodes in the input document to the templates in the stylesheet. When it finds a match, it instantiates the template and adds the result to the output tree.

The biggest difference between XSLT and traditional programming languages is that the input document drives the flow of the program rather than the stylesheet controlling it explicitly. When designing an XSLT stylesheet, you concentrate on which input constructs map to which output constructs rather than on how or when the processor reads the input and generates the output.

In some sense, XSLT is a push model like SAX rather than a pull model like DOM. This approach is initially uncomfortable for programmers accustomed to more procedural languages. However, it does have the advantage of being much more robust against unexpected changes in the structure of the input data. An XSLT transform rarely fails completely just because an expected element is missing or misplaced or because an unexpected, invalid element is encountered.

Tip

If you are concerned about the exact structure of the input data and want to respond differently if it’s not precisely correct (for instance, an XML-RPC server should respond to a malformed request with a fault document rather than a best-guess) validate the documents with a DTD or a schema before transforming them. XSLT doesn’t provide any means to do this. However, you can implement this in Java in a separate layer before deciding whether to pass the input document to the XSLT processor for transformation.

Template Rules

An XSLT stylesheet contains examples of what belongs in the output document, roughly one example for each significantly different construct that exists in the input documents. It also contains instructions telling the XSLT processor how to convert input nodes into the example output nodes. The XSLT processor uses those examples and instructions to convert nodes in the input documents to nodes in the output document.

Examples and instructions are written as template rules. Each template rule has a pattern and a template. The template rule is represented by an xsl:template element. The customary prefix xsl is bound to the namespace URI http://www.w3.org/1999/XSL/Transform, and as usual the prefix can change as long as the URI remains the same. The pattern, a limited form of an XPath expression, is stored in this element’s match attribute. The contents of the xsl:template element form the template. For example, this is a template rule that matches methodCall elements and responds with a template consisting of a single methodResponse element:

<xsl:template match="methodCall">
  <methodResponse>
    <params>
      <param>
        <value><string>Hello</string></value>
      </param>
    </params>
  </methodResponse>
</xsl:template>

Stylesheets

A complete XSLT stylesheet is a well-formed XML document. The root element of this document is xsl:stylesheet which has a version attribute with the value 1.0. In practice, stylesheets normally contain multiple template rules matching different kinds of input nodes, but for now Example 17.1 shows one that just contains one template rule:

Example 17.1. An XSLT stylesheet for XML-RPC request documents

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <xsl:template match="methodCall">
    <methodResponse>
      <params>
        <param>
          <value><string>Hello</string></value>
        </param>
      </params>
    </methodResponse>
  </xsl:template>

</xsl:stylesheet>

Applying this stylesheet to any XML-RPC request document produces the following result:

<?xml version="1.0" encoding="utf-8"?><methodResponse><params>
<param><value><string>Hello</string></value></param></params>
</methodResponse>

The template in Example 17.1’s template rule consists exclusively of literal result elements and literal data that are copied directly to the output document from the stylesheet. It also contains some white-space only text nodes, but by default an XSLT processor strips these out. (You can keep the white space by adding an xml:space="preserve" attribute to the xsl:template element if you want.) A template can also contain XSLT instructions that copy data from the input document to the output document or create new data algorithmically.

Taking the Value of a Node

Perhaps the most common XSLT instruction is xsl:value-of. This returns the XPath string-value of an object selected by an XPath expression. For example, the value of an element is the concatenation of all the character data but none of the markup contained between the element’s start-tag and end-tag. Each xsl:value-of element has a select attribute whose value contains an XPath expression identifying the object to take the value of. For example, this xsl:value-of element takes the value of the root methodCall element:

<xsl:value-of select="/methodCall" />

This xsl:value-of element takes the value of the root int element further down the tree:

<xsl:value-of select="/methodCall/params/value/int" />

This xsl:value-of element uses a relative location path. It calculates the string-value of the int child of the value child of the params child of the context node (normally the node matched by the containing template):

<xsl:value-of select="params/value/int" />

The xsl:value-of element can calculate the value of any of the four XPath data types (number, boolean, string, and node-set). For example, this expression calculates the value of e times π:

<xsl:value-of select="2.71828 * 3.141592" />

In fact, you can use absolutely any legal XPath expression in the select attribute. This xsl:value-of element multiplies the number in the int element by ten and returns it:

<xsl:value-of select="10 * params/value/int" />

In all cases the value of an object is the XPath string-value that would be returned by the XPath string() function.

When xsl:value-of is used in a template, the context node is a node matched by the template and for which the template is being instantiated. The template in Example 17.2 copies the string-value of the value element in the input document to the string element in the output document.

Example 17.2. An XSLT stylesheet that echoes XML-RPC requests

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:template match="methodCall" xml:space="preserve">
<methodResponse>
  <params>
    <param>
      <value>
        <string>
          <xsl:value-of select="params/param/value" />
        </string>
      </value>
    </param>
  </params>
</methodResponse>
</xsl:template>

</xsl:stylesheet>

If this stylesheet is applied to the XML document in Example 17.3, then the result is the XML document shown in Example 17.4. There are a number of GUI, command line, and server side programs that will do this, though our interest is going to be in integrating XSLT stylesheets into Java programs so I’m going to omit the details of exactly how this takes place for the moment. I’ll pick it up again in the next section.

Example 17.3. An XML-RPC request document

<?xml version="1.0"?>
<methodCall>
  <methodName>calculateFibonacci</methodName>
  <params>
    <param>
      <value><int>10</int></value>
    </param>
  </params>
</methodCall>

Example 17.4. An XML-RPC response document

<?xml version="1.0" encoding="utf-8"?>
<methodResponse>
  <params>
    <param>
      <value>
        <string>
          10
        </string>
      </value>
    </param>
  </params>
</methodResponse>

The white space in the output is a little prettier here than in the last example because I used an xml:space="preserve" attribute in the stylesheet. More importantly, the content of the string element has now been copied from the input document. The output is a combination of literal result data from the stylesheet and information read from the transformed XML document.

Applying Templates

Probably the single most important XSLT instruction is the one that tells the processor to continue processing other nodes in the input document and instantiate their matching templates. This instruction is the xsl:apply-templates element. Its select attribute contains an XPath expression identifying the nodes to apply templates to. The currently matched node is the context node for this expression. For example, this template matches methodCall elements. However, rather than outputting a fixed response, it generates a methodResponse element whose contents are formed by instantiating the template for each params child element in turn:

<xsl:template match="methodCall">
  <methodResponse>
    <xsl:apply-templates select="child::params"/>
  </methodResponse>
</xsl:template>

The complete output this template rule generates depends on what the template rule for the params element does. That template rule may further apply templates to its own param children like this:

<xsl:template match="params">
  <params>
    <xsl:apply-templates select="child::param"/>
  </params>
</xsl:template>

The param template rule may apply templates to its value child like this:

<xsl:template match="param">
  <param>
    <xsl:apply-templates select="child::value"/>
  </param>
</xsl:template>

The value template rule may apply templates to all its element children whatever their type using the * wild card:

<xsl:template match="value">
  <value>
    <xsl:apply-templates select="child::*"/>
  </value>
</xsl:template>

Finally one template rule may take the value of all the possible children of value at once using the union operator |:

<xsl:template match="int | i4 | string | boolean | double
 | dateTime.iso8601 | base64 | struct">
  <string>
    <xsl:value-of select="."/>
  </string>
</xsl:template>

This example descended straight down the expected tree, and mostly copied the existing markup. However, XSLT is a lot more flexible and can move in many different directions at any point. Templates can skip nodes, move to preceding or following siblings, reprocess previously processed nodes, or move along any of the axes defined in XPath.

The Default Template Rules

XSLT defines several default template rules that are used when no explicit rule matches a node. The first such rule applies to the root node and to element nodes. It simply applies templates to the children of that node but does not specifically generate any output:

<xsl:template match="*|/">
  <xsl:apply-templates/>
</xsl:template>

This allows the XSLT processor to walk the tree from top to bottom by default, unless told to do something else by other templates. The default templates are overridden by any explicit templates.

The second default rule applies to text and attribute nodes. It copies the value of each of these nodes into the output tree:

<xsl:template match="text()|@*">
  <xsl:value-of select="."/>
</xsl:template>

Together, these rules have the effect of copying the text contents of an element or document to the output but deleting the markup structure. Of course you can change this behavior by overriding the built-in template rules with your own template rules.

Selection

Adding XPath predicates to match patterns and select expressions offers a lot of the if-then functionality you need. However, sometimes that’s not quite enough. In that case, you can take advantage of the xsl:if and xsl:choose elements.

xsl:if

The xsl:if instruction enables the stylesheet to decide whether or not to do something. It contains a template. If the XPath expression in the test attribute evaluates to true, then the template is instantiated and added to the result tree. If the XPath expression evaluates to false, then it isn’t. If the XPath expression evaluates to something that isn’t a boolean, then it is converted to true or false using the XPath boolean() function described in the previous chapter. That is, 0 and NaN are false. All other numbers are true. Empty strings and node-sets are false. Non-empty strings and node-sets are true.

For example, when evaluating an XML-RPC request you might want to check that the request document indeed adheres to the specification, or at least that it’s close enough for what you need it for. This requires checking that the root element is methodCall, that the root element has exactly one methodName child and one params child, and so forth. Here’s an XPath expression that checks for various violations of XML-RPC syntax (Remember that according to XPath empty node-sets are false and non-empty node-sets are true.):

count(/methodCall/methodName) != 1
 or count(/methodCall/params) != 1
 or not(/methodCall/params/param/value)

I could check considerably more than this, but this suffices for an example. Now we can use this XPath expression in an xsl:if test inside the template for the root node. If the test succeeds (that is, the request document is incorrect), then the xsl:message instruction terminates the processing:

<xsl:template match="/">
  <xsl:if test="count(/methodCall/methodName) != 1
                 or count(/methodCall/params) != 1
                 or not(/methodCall/params/param/value)">
    <xsl:message terminate="yes">
      The request document is invalid.
    </xsl:message>
  </xsl:if>
  <xsl:apply-templates select="child::methodCall"/>
</xsl:template>

The exact behavior of the xsl:message instruction is processor dependent. The message might be delivered by printing it on System.out, writing it into a log file, or popping up a dialog box. Soon, you’ll see how to generate a fault document instead.

There is no xsl:else or xsl:else-if instruction. To choose from multiple alternatives, use the xsl:choose instruction instead.

xsl:choose

The xsl:choose instruction selects from multiple alternative templates. It contains one or more xsl:when elements, each of which contains a test attribute and a template. The first xsl:when element whose test attribute evaluates to true is instantiated. The others are ignored. There may also be an optional final xsl:otherwise element whose template is instantiated only if all the xsl:when elements are false.

For example, when an XML-RPC request is well-formed but syntactically incorrect, the server should respond with a fault document. This template tests for a number of possible problems with an XML-RPC request and only processes it if none of the problems arise. Otherwise it emits an error message:

<xsl:template match="/">
  <xsl:choose>
    <xsl:when test="not(/methodCall/methodName)">
       Missing methodName
    </xsl:when>
    <xsl:when test="count(/methodCall/methodName) &gt; 1">
      Multiple methodNames
    </xsl:when>
    <xsl:when test="count(/methodCall/params) &gt; 1">
       Multiple params elements
    </xsl:when>
    <xsl:otherwise>
      <xsl:apply-templates select="child::methodCall"/>
    </xsl:otherwise>
  </xsl:choose>
</xsl:template>

XSLT does not have any real exception handling or error reporting mechanism. In the worst case, the processor simply gives up and prints an error message on the console. This template, like every other template, is instantiated to create a node-set. Depending on which conditions are true, different nodes will be in that set. If there is an error condition, this set will contain a single text node with the contents, “Missing methodName”, “Multiple methodNames”, or “Multiple params elements”. Otherwise it will contain whatever nodes are created by applying templates to the methodCall child element. But in both cases a node-set is returned that is inserted into the output document.

Calling Templates by Name

There is a second way a template can be instantiated. As well as matching a node in the input document, a template can be called by name using the xsl:call-template element. Parameters can be passed to such templates. Templates can even be called recursively. Indeed, it is recursion that makes XSLT Turing-complete.

For example, here’s a template named faultResponse that generates a complete XML-RPC fault document when invoked::

<xsl:template name="faultResponse">
  <methodResponse>
    <fault>
      <value>
        <struct>
          <member>
            <name>faultCode</name>
            <value><int>0</int></value>
          </member>
          <member>
            <name>faultString</name>
            <value><string>Invalid request document</string></value>
          </member>
        </struct>
      </value>
    </fault>
  </methodResponse>
</xsl:template>

The xsl:call-template element applies a named template to the context node. For example, earlier you saw a root node template that terminated the processing if it detected an invalid document. Now it can call the fault template instead:

<xsl:template match="/">
  <xsl:choose>
    <xsl:when test="count(/methodCall/methodName) != 1
                 or count(/methodCall/params) != 1
                 or not(/methodCall/params/param/value)">
      <xsl:call-template name="faultResponse"/>
    </xsl:when>
    <xsl:otherwise>
      <xsl:apply-templates select="child::methodCall"/>
    </xsl:otherwise>
  </xsl:choose>
</xsl:template>

Named templates can factor out common code that’s used in multiple places through top-down design, just as a complicated algorithm in a Java program may be broken into multiple methods rather than one large monolithic method. Indeed some large stylesheets including the DocBook XSL stylesheets that produced this book do use named templates for this purpose. However, named templates become even more important when you add parameters and recursion to the mix.

Passing Parameters to Templates

Each template rule can have any number of parameters represented as xsl:param elements. These appear inside the xsl:template element before the template itself. Each xsl:param element has a name attribute and an optional select attribute. The select attribute provides a default value for that parameter when the template is invoked but can be overridden. If the select attribute is omitted, then the default value for the parameter is set by the contents of the xsl:param element. (For a non-overridable variable, you can use a local xsl:variable element instead.)

For example, the parameters in this fault template specify the faultCode and the faultString. The default fault code is 0. The default fault string is Unspecified Error.

<xsl:template name="faultResponse">
  <xsl:param name="err_code" select="0"/>
  <xsl:param name="err_message">Unspecified Error</xsl:param>
  <methodResponse>
    <fault>
      <value>
        <struct>
          <member>
            <name>faultCode</name>
            <value>
              <int><xsl:value-of select="$err_code"/></int>
            </value>
          </member>
          <member>
            <name>faultString</name>
            <value>
              <string>
                <xsl:value-of select="$err_message"/>
              </string>
            </value>
          </member>
        </struct>
      </value>
    </fault>
  </methodResponse>
</xsl:template>

XSLT is weakly typed. There is no type attribute on the xsl:param element. You can pass in pretty much any object as the value of one of these parameters. If you use such a variable in a place where an item of that type can’t be used and can’t be converted to the right type, then the processor will stop and report an error.

The xsl:call-template element can provide values for each of the named parameters using xsl:with-param child elements, or it can accept the default values specified by the xsl:param elements. For example, this template rule for the root node uses different error codes and messages for different problems:

<xsl:template match="/">
  <xsl:choose>
    <xsl:when test="not(/methodCall/methodName)">
      <xsl:call-template name="faultResponse">
        <xsl:with-param name="err_code" select="1" />
        <xsl:with-param name="err_message">
          Missing methodName
        </xsl:with-param>
      </xsl:call-template>
    </xsl:when>
    <xsl:when test="count(/methodCall/methodName) &gt; 1">
      <xsl:call-template name="faultResponse">
        <xsl:with-param name="err_code" select="1" />
        <xsl:with-param name="err_message">
          Multiple method names
        </xsl:with-param>
      </xsl:call-template>
    </xsl:when>
    <xsl:when test="count(/methodCall/params) &gt; 1">
      <xsl:call-template name="faultResponse">
        <xsl:with-param name="err_code" select="2" />
        <xsl:with-param name="err_message">
          Multiple params elements
        </xsl:with-param>
      </xsl:call-template>
    </xsl:when>
  <!-- etc. -->
    <xsl:otherwise>
      <xsl:apply-templates select="child::methodCall"/>
    </xsl:otherwise>
  </xsl:choose>
</xsl:template>

I’m not sure I would always recommend this approach for validation. Most of the time writing a schema is easier. However, this technique can be used to verify things a schema can’t. For example, it could test that a value element contains either an ASCII string or a type element such as int, but not a type element and an ASCII string.

Recursion

The ability for a template to call itself (recursion) is the final ingredient of a fully Turing-complete language. For example, here’s a template that implements the factorial function:

<xsl:template name="factorial">
  <xsl:param name="arg" select="0"/>
  <xsl:param name="return_value" select="1"/>
  <xsl:choose>
    <xsl:when test="$arg = 0">
      <xsl:value-of select="$return_value"/>
    </xsl:when>
    <xsl:when test="$arg &gt; 0">
      <xsl:call-template name="factorial">
        <xsl:with-param name="arg" select="$arg - 1"/>
        <xsl:with-param name="return_value"
                        select="$return_value * $arg"/>
      </xsl:call-template>
    </xsl:when>
    <xsl:when
      test="$arg &lt; 0">Error: function undefined!</xsl:when>
  </xsl:choose>
</xsl:template>

The factorial template has two arguments, $arg and $return_value. $arg is the number whose factorial the client wants, and must be passed as parameter the first time this template is invoked. $return_value is initially 1. When $arg reaches zero, the template returns $return_value. However, if $arg is not 0, the template decrements $arg by 1, multiplies $return_value by the current value of $arg, and calls itself again.

Functional languages like XSLT do not allow variables to change their values or permit side effects. This can seem a little strange to programmers accustomed to imperative languages like Java. The key is to remember that almost any task a loop performs in Java, recursion performs in XSLT. For example, consider the most basic CS101 problem, printing out the integers from 1 to 10. In Java it’s a simple for loop:

for (int i=1; i <= 10; i++) {
  System.out.print(i);
}

However, in XSLT you’d use recursion in this fashion:

<xsl:template name="CS101">
  <xsl:param name="index" select="1"/>
  <xsl:if test="$index &lt;= 10">
    <xsl:value-of select="$index"/>
    <xsl:call-template name="CS101">
      <xsl:with-param name="index" select="$index + 1"/>
    </xsl:call-template>
  </xsl:if>
</xsl:template>

Similar recursive techniques can be used for other looping operations such as sums, averages, sorting, and more. Neither iteration nor recursion is mathematically better or fundamentally faster than the other. They both produce the same results in the end.

Note

The XSLT solution is more complex and less obvious than the Java equivalent. However, that has more to do with XSLT’s XML syntax than with recursio`n itself. In Java the same operation could be written recursively like this:

public void fakeLoop(int i) {
  System.out.print(i);
  if (i < 10) fakeLoop(i++);
}

In fact, it has been proven that given sufficient memory any recursive algorithm can be transformed into an iterative one and vice versa.[1]

Let’s look at a more complex example. Example 17.5 is a simple XSLT stylesheet that reads input XML-RPC requests in the form of Example 17.3 and converts them into output XML-RPC responses.

Example 17.5. An XSLT stylesheet that calculates Fibonacci numbers

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <xsl:template match="/">
    <xsl:choose>
      <!-- Basic sanity check on the input -->
      <xsl:when
        test="count(methodCall/params/param/value/int) = 1">
        <xsl:apply-templates select="child::methodCall"/>
      </xsl:when>
      <xsl:otherwise>
        <!-- Sanity check failed -->
        <xsl:call-template name="faultResponse"/>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>

  <xsl:template match="methodCall">
    <methodResponse>
      <params>
        <param>
          <value>
            <xsl:apply-templates
              select="params/param/value/int"/>
          </value>
        </param>
      </params>
    </methodResponse>
  </xsl:template>

  <xsl:template match="int">
    <int>
      <xsl:call-template name="calculateFibonacci">
        <xsl:with-param name="index" select="number(.)"/>
      </xsl:call-template>
    </int>
  </xsl:template>

  <xsl:template name="calculateFibonacci">
    <xsl:param name="index"/>
    <xsl:param name="low"  select="1"/>
    <xsl:param name="high" select="1"/>
    <xsl:choose>
      <xsl:when test="$index &lt;= 1">
        <xsl:value-of select="$low"/>
      </xsl:when>
      <xsl:otherwise>
        <xsl:call-template name="calculateFibonacci">
          <xsl:with-param name="index" select="$index - 1"/>
          <xsl:with-param name="low"   select="$high"/>
          <xsl:with-param name="high"  select="$high + $low"/>
        </xsl:call-template>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>

  <xsl:template name="faultResponse">
    <xsl:param name="err_code"    select="0" />
    <xsl:param name="err_message" select="'Unspecified Error'"/>
    <methodResponse>
      <fault>
        <value>
          <struct>
            <member>
              <name>faultCode</name>
              <value>
                <int><xsl:value-of select="$err_code"/></int>
              </value>
            </member>
            <member>
              <name>faultString</name>
              <value>
                <string>
                  <xsl:value-of select="$err_message"/>
                </string>
              </value>
            </member>
          </struct>
        </value>
      </fault>
    </methodResponse>
  </xsl:template>

</xsl:stylesheet>

However, although XSLT is Turing-complete in a theoretical sense, practically it is missing a lot of the functionality you’d expect from a modern programming language such as mathematical functions, I/O, and network access. To actually use this stylesheet as an XML-RPC server, we need to wrap it up in a Java program that can provide all this.

[1] See, for example, Robert Sedgewick, Removing Recursion, in Algorithms in C++, (Addison-Wesley, 1992) pp. 61-65

Prev | Up | Next
Summary | Home | TrAX
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 20, 2002


TrAX
Prev Chapter 17. XSLT Next

TrAX

TrAX, the Transformations API for XML, is a Java API for performing XSLT transforms. It is sufficiently parser-independent that it can work with many different XSLT processors including Xalan and SAXON. It is sufficiently model-independent that it can transform to and from XML streams, SAX event sequences, and DOM and JDOM trees.

TrAX is a standard part of JAXP, and is bundled with Java 1.4 and later. Furthermore, most current XSLT processors written in Java support TrAX including Xalan-J 2.x, jd.xslt, LotusXSL, and Saxon. The specific implementation included with Java 1.4.0 is Xalan-J 2.2D10.

Note

Annoyingly, the Xalan-J classes included in Java 1.4 are zipped into the rt.jar archive so it’s hard to replace them with a less-buggy release version of Xalan. It can be done, but you have to put the xalan.jar file in your $JAVA_HOME/lib/endorsed directory rather than in the normal jre/lib/ext directory. The exact location of $JAVA_HOME varies from system to system, but it’s probably something like C:\j2sdk1.4.0 on Windows. None of this is an issue with Java 1.3 and earlier, which don’t bundle these classes. On these systems you just need to install whatever jar files your XSLT engine vendor provides in the usual locations, the same as you would any other third party library.

There are four main classes and interfaces in TrAX that you need to use, all in the javax.xml.transforms package:

Transformer

The class that represents the style sheet. It transforms a Source into a Result.

TransformerFactory

The class that represents the XSLT processor. This is a factory class that reads a stylesheet to produce a new Transformer.

Source

The interface that represents the input XML document to be transformed, whether presented as a DOM tree, an InputStream, or a SAX event sequence.

Result

The interface that represents the XML document produced by the transformation, whether generated as a DOM tree, an OutputStream, or a SAX event sequence.

To transform an input document into an output document follow these steps:

  1. Load the TransformerFactory with the static TransformerFactory.newInstance() factory method.

  2. Form a Source object from the XSLT stylesheet.

  3. Pass this Source object to the factory’s newTransformer() factory method to build a Transformer object.

  4. Build a Source object from the input XML document you wish to transform.

  5. Build a Result object for the target of the transformation.

  6. Pass both the source and the result to the Transformer object’s transform() method.

Steps four through six can be repeated for as many different input documents as you want. You can reuse the same Transformer object repeatedly in series, though you can’t use it in multiple threads in parallel.

For example, suppose you want to use the Fibonacci stylesheet in Example 17.5 to implement a simple XML-RPC server. The request document will arrive on an InputStream named in and be returned on an OutputStream named out. Therefore, we’ll use javax.xml.transform.stream.StreamSource as the Source for the input document and javax.xml.transform.stream.StreamResult as the Result for the output document. The stylesheet itself will also be assumed to live at the relative URL FibonacciXMLRPC.xsl, and will also be loaded into a javax.xml.transform.stream.StreamSource. This code fragment performs that transform:

try {
  TransformerFactory xformFactory
   = TransformerFactory.newInstance();
  Source xsl = new StreamSource("FibonacciXMLRPC.xsl");
  Transformer stylesheet = xformFactory.newTransformer(xsl);

  Source request  = new StreamSource(in);
  Result response = new StreamResult(out);
  stylesheet.transform(request, response);
}
catch (TransformerException e) {
  System.err.println(e);
}

Thread Safety

Neither TransformerFactory nor Transformer is guaranteed to be thread-safe. If your program is multi-threaded, the simplest solution is just to give each separate thread its own TransformerFactory and Transformer objects. However, this can be expensive, especially if you frequently reuse the same large stylesheet, since it will need to be read from disk or the network and parsed every time you create a new Transformer object. There is also likely to be some overhead in building the processor’s internal representation of an XSLT stylesheet from the parsed XML tree.

An alternative is to ask the TransformerFactory to build a Templates object instead. The Templates class represents the parsed stylesheet. You can then ask the Templates class to give you as many separate Transformer objects as you need, each of which can be created very quickly by copying the processor’s in-memory data structures rather than by reparsing the entire stylesheet from disk or the network. The Templates class itself can be safely used across multiple threads.

For example, you might begin loading and compiling the stylesheet like this:

  TransformerFactory xformFactory
   = TransformerFactory.newInstance();
  Source xsl = new StreamSource("FibonacciXMLRPC.xsl");
  Templates templates = xformFactory.newTemplates(xsl);

Then later in a loop you’d repeatedly load documents and transform them like this:

while (true) {
  InputStream  in   = getNextDocument();
  OutputStream out  = getNextTarget();
  Source request    = new StreamSource(in);
  Result response   = new StreamResult(out);
  Transformer transformer = templates.newTransformer();
  transformer.transform(request, response);
}

Since the thread-unsafe Transformer object is local to the while loop, references to it don’t escape into other threads. This prevents the transform() method from being called concurrently. The Templates object may be shared among multiple threads. However, it is thread safe so this isn’t a problem. Furthermore, all the time-consuming work is done when the Templates object is created. Calling templates.newTransformer() is very quick by comparison.

This technique is particularly important in server environments where the transform may be applied to thousands of different input documents with potentially dozens being processed in parallel in separate threads at the same time. Example 17.6 demonstrates with yet another variation of the Fibonacci XML-RPC servlet. This is the first variation that does not implement the SingleThreadModel interface. It can safely run in multiple threads simultaneously.

Example 17.6. A servlet that uses TrAX and XSLT to respond to XML-RPC requests

import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;
import javax.xml.transform.*;
import javax.xml.transform.stream.*;

public class FibonacciXMLRPCXSLServlet extends HttpServlet {

  private Templates stylesheet;

  // Load the stylesheet
  public void init() throws ServletException {

    try {
      TransformerFactory xformFactory
       = TransformerFactory.newInstance();
      Source source   = new StreamSource("FibonacciXMLRPC.xsl");
      this.stylesheet = xformFactory.newTemplates(source);
    }
    catch (TransformerException e) {
      throw new ServletException(
       "Could not load the stylesheet", e);
    }

  }

  // Respond to an XML-RPC request
  public void doPost(HttpServletRequest servletRequest,
   HttpServletResponse servletResponse)
   throws ServletException, IOException {

    servletResponse.setContentType("text/xml; charset=UTF-8");

    try {
      InputStream in  = servletRequest.getInputStream();
      Source source   = new StreamSource(in);
      PrintWriter out = servletResponse.getWriter();
      Result result   = new StreamResult(out);
      Transformer transformer = stylesheet.newTransformer();
      transformer.transform(source, result);
      servletResponse.flushBuffer();
      out.flush();
      out.println();
    }
    catch (TransformerException e) {
      // If we get an exception at this point, it's too late to
      // switch over to an XML-RPC fault.
      throw new ServletException(e);
    }

  }

}

The init() method simply loads the stylesheet that will transform requests into responses. The doPost() method reads the request and returns the response. The Source is a StreamSource. The result is a StreamResult.

I’m not sure I would recommend this as the proper design for a servlet of this nature. The XSLT transform comes with a lot of overhead. At the least, I would definitely recommend doing the math in Java since XSLT is not optimized for this sort of work. Still, I’m quite impressed with the simplicity and robustness of this code. The thread safety is just the first benefit. Shifting the XML generation into an XSLT document makes the whole program a lot more modular. It’s easy to change the expected input or output format without even recompiling the servlet.

Locating Transformers

The javax.xml.transform.TransformerFactory Java system property determines which XSLT engine TrAX uses. Its value is the fully qualified name of the implementation of the abstract javax.xml.transform.TransformerFactory class. Possible values of this property include:

  • Saxon 6.x: com.icl.saxon.TransformerFactoryImpl

  • Saxon 7.x: net.sf.saxon.TransformerFactoryImpl

  • Xalan: org.apache.xalan.processor.TransformerFactoryImpl

  • jd.xslt: jd.xml.xslt.trax.TransformerFactoryImpl

  • Oracle: oracle.xml.jaxp.JXSAXTransformerFactory

This property can be set in all the usual ways a Java system property can be set. TrAX picks from them in this order:

  1. System.setProperty( "javax.xml.transform.TransformerFactory", "classname")

  2. The value specified at the command line using the -Djavax.xml.transform.TransformerFactory=classname option to the java interpreter

  3. The class named in the lib/jaxp.properties properties file in the JRE directory, in a line like this one:

    javax.xml.transform.TransformerFactory=classname

  4. The class named in the META-INF/services/javax.xml.transform.TransformerFactory file in the JAR archives available to the runtime

  5. Finally, if all of the above options fail, TransformerFactory.newInstance() returns a default implementation. In Sun’s JDK 1.4, this is Xalan 2.2d10.

The xml-stylesheet processing instruction

XML documents may contain an xml-stylesheet processing instruction in their prologs that specifies the stylesheet to apply to the XML document. At a minimum, this has an href pseudo-attribute specifying the location of the stylesheet to apply and a type pseudo-attribute specifying the MIME media type of the stylesheet. For XSLT stylesheets the proper type is application/xml. For example, this xml-stylesheet processing instruction indicates the XSLT stylesheet found at the relative URL docbook-xsl-1.50.0/fo/docbook.xsl:

<?xml-stylesheet href="docbook-xsl-1.50.0/fo/docbook.xsl"
                 type="application/xml"?>

Note

Contrary to what some other books will tell you, there is no such MIME media type as text/xsl, nor is it correct to use it as the value of the type pseudo-attribute. This alleged type is a figment of Microsoft’s imagination. It has never been registered with the IANA as MIME types must be. It is not endorsed by the relevant W3C specifications for XSLT and attaching stylesheets to XML documents, and it is not likely to be in the future.

Official registration of an XSLT specific media type application/xml+xslt has begun, and this type may be used in the future to distinguish XSLT stylesheets from other kinds of XML document. However, the registration has not been completed at the time of this writing.

This processing instruction is a hint. It is only a hint. Programs are not required to use the stylesheet the document indicates. They are free to choose a different transform, multiple transforms, or no transform at all. Indeed, the purpose of this processing instruction is primarily browser display. Programs doing something other than loading the document into a browser for a human to read will likely want to use their own XSLT transforms for their own purposes.

Besides the required href and type pseudo-attributes, the xml-stylesheet processing instruction can also have up to four other optional pseudo-attributes:

alternate

no if this stylesheet is the primary stylesheet for the document; yes if it isn’t. The default is no.

media

A string indicating in which kinds of environments this stylesheet should be used. Possible values include screen (the default), tty, tv, projection, handheld, print, braille, aural, and all.

charset

The character encoding of the stylesheet; e.g. ISO-8859-1, UTF-8, or SJIS.

title

A name for the stylesheet.

For example, these xml-stylesheet processing instructions point at two different XSLT stylesheets, one intended for print and found at the relative URL docbook-xsl-1.50.0/fo/docbook.xsl and the other intended for onscreen display and found at docbook-xsl-1.50.0/html/docbook.xsl. Both are the primary stylesheet for their media.

<?xml-stylesheet href="docbook-xsl-1.50.0/fo/docbook.xsl"
                 type="application/xml"
                 media="print"
                 title="XSL-FO"
                 encoding="UTF-8"
                 alternate="no"?>
<?xml-stylesheet href="docbook-xsl-1.50.0/html/docbook.xsl"
                 type="application/xml"
                 media="screen"
                 title="HTML"
                 encoding="UTF-8"
                 alternate="no"?>

The TransformerFactory class has a getAssociatedStylesheet() method that loads the stylesheet indicated by such a processing instruction:

public abstract Source getAssociatedStylesheet(Source xmlDocument, String media, String title, String charset)
    throws TransformerConfigurationException;

This method reads the XML document indicated by the first argument, and looks in its prolog for the stylesheet that matches the criteria given in the other three arguments. If any of these are null, it ignores that criterion. The method then loads the stylesheet matching the criteria into a JAXP Source object and returns it. You can use the TransformerFactory.newTransformer() object to convert this Source into a Transformer object. For example, this code fragment attempts to transform the document read from the InputStream in according to an xml-stylesheet processing instruction for print media found in that document’s prolog. The title and encoding of the stylesheet are not considered, and thus set to null.

// The InputStream in contains the XML document to be transformed
try {
  Source inputDocument = new StreamSource(in);
  TransformerFactory xformFactory
   = TransformerFactory.newInstance();
  Source xsl = xformFactory.getAssociatedStyleSheet(
   inputDocument, "print", null, null);
  Transformer stylesheet = xformFactory.newTransformer(xsl);

  Result outputDocument = new StreamResult(out);
  stylesheet.transform(inputDocument, outputDocument);
}
catch (TransformerConfigurationException e) {
  System.err.println("Problem with the xml-stylesheet processing instruction");
}
catch (TransformerException e) {
  System.err.println("Problem with the stylesheet");
}

A TransformerConfigurationException is thrown if there is no xml-stylesheet processing instruction pointing to an XSLT stylesheet matching the specified criteria.

Features

Not all XSLT processors support exactly the same set of capabilities, even within the limits defined by XSLT 1.0. For example, some processors can only transform DOM trees, whereas others may require a sequence of SAX events, and still others may only be able to work with raw streams of text. TrAX uses URI-named features to indicate which of the TrAX classes any given implementation supports. It defines eight standard features as unresolvable URL strings, each of which is also available as a named constant in the relevant TrAX class:

  • StreamSource.FEATURE: http://javax.xml.transform.stream.StreamSource/feature

  • StreamResult.FEATURE: http://javax.xml.transform.stream.StreamResult/feature

  • DOMSource.FEATURE: http://javax.xml.transform.dom.DOMSource/feature

  • DOMResult.FEATURE: http://javax.xml.transform.dom.DOMResult/feature

  • SAXSource.FEATURE: http://javax.xml.transform.dom.SAXSource/feature

  • SAXResult.FEATURE: http://javax.xml.transform.dom.SAXResult/feature

  • SAXTransformerFactory.FEATURE: http://javax.xml.transform.sax.SAXTransformerFactory/feature

  • SAXTransformerFactory.FEATURE_XMLFILTER: http://javax.xml.transform.sax.SAXTransformerFactory/feature/xmlfilter

Note

These URLs are just identifiers like namespace URLs. They do not need to be and indeed cannot be resolved. A system does not need to be connected to the Internet to use a transformer that supports these features.

The boolean values of these features for the current XSLT engine can be tested with the getFeature() method in the TransformerFactory class:

public abstract boolean getFeature(String Name);

There’s no corresponding setFeature() method because a TrAX feature reflects the nature of the underlying parser. Unlike, a SAX feature it is not something you can just turn on or off with a switch. Either a processor supports DOM input or it doesn’t. Either a processor supports SAX output or it doesn’t, and so on.

Example 17.7 is a simple program that can be used to test an XSLT processor’s support for the standard JAXP 1.1 features.

Example 17.7. Testing the availability of TrAX features

import javax.xml.transform.*;
import javax.xml.transform.dom.*;
import javax.xml.transform.stream.*;
import javax.xml.transform.sax.*;

public class TrAXFeatureTester {

  public static void main(String[] args) {

    TransformerFactory xformFactory
     = TransformerFactory.newInstance();

    String name = xformFactory.getClass().getName();

    if (xformFactory.getFeature(DOMResult.FEATURE)) {
      System.out.println(name + " supports DOM output.");
    }
    else {
      System.out.println(name + " does not support DOM output.");
    }
    if (xformFactory.getFeature(DOMSource.FEATURE)) {
      System.out.println(name + " supports DOM input.");
    }
    else {
      System.out.println(name + " does not support DOM input.");
    }

    if (xformFactory.getFeature(SAXResult.FEATURE)) {
      System.out.println(name + " supports SAX output.");
    }
    else {
      System.out.println(name + " does not support SAX output.");
    }
    if (xformFactory.getFeature(SAXSource.FEATURE)) {
      System.out.println(name + " supports SAX input.");
    }
    else {
      System.out.println(name + " does not support SAX input.");
    }

    if (xformFactory.getFeature(StreamResult.FEATURE)) {
      System.out.println(name + " supports stream output.");
    }
    else {
      System.out.println(name + " does not support stream output.");
    }
    if (xformFactory.getFeature(StreamSource.FEATURE)) {
      System.out.println(name + " supports stream input.");
    }
    else {
      System.out.println(name + " does not support stream input.");
    }

    if (xformFactory.getFeature(SAXTransformerFactory.FEATURE)) {
      System.out.println(name + " returns SAXTransformerFactory "
       + "objects from TransformerFactory.newInstance().");
    }
    else {
      System.out.println(name
       + " does not use SAXTransformerFactory.");
    }
    if (xformFactory.getFeature(SAXTransformerFactory.FEATURE_XMLFILTER)) {
      System.out.println(
       name + " supports the newXMLFilter() methods.");
    }
    else {
      System.out.println(
       name + " does not support the newXMLFilter() methods.");
    }

  }

}

Here’s the results of running this program against Saxon 6.5.1:

C:\XMLJAVA>java -Djavax.xml.transform.TransformerFactory=
com.icl.saxon.TransformerFactoryImpl TrAXFeatureTester
com.icl.saxon.TransformerFactoryImpl supports DOM output.
com.icl.saxon.TransformerFactoryImpl supports DOM input.
com.icl.saxon.TransformerFactoryImpl supports SAX output.
com.icl.saxon.TransformerFactoryImpl supports SAX input.
com.icl.saxon.TransformerFactoryImpl supports stream output.
com.icl.saxon.TransformerFactoryImpl supports stream input.
com.icl.saxon.TransformerFactoryImpl returns
 SAXTransformerFactory objects from
 TransformerFactory.newInstance().
com.icl.saxon.TransformerFactoryImpl supports the newXMLFilter()
 methods.

As you can see Saxon supports all eight features. Xalan also supports all eight features.

XSLT Processor Attributes

Some XSLT processors provide non-standard, custom attributes that control their behavior. Like features, these are also named via URIs. For example, Xalan-J 2.3 defines these three attributes:

http://apache.org/xalan/features/optimize

By default, Xalan rewrites stylesheets in an attempt to optimize them (similar to the behavior of an optimizing compiler for Java or other languages). This can confuse tools that need direct access to the stylesheet such as XSLT profilers and debuggers. If you’re using such a tool with Xalan, you should set this attribute to false.

http://apache.org/xalan/features/incremental

Setting this feature to true allows Xalan to begin producing output before it has finished processing the entire input document. This may cause problems if an error is detected late in the process, but it shouldn’t be a big problem in fully debugged and tested environments.

http://apache.org/xalan/features/source_location

Setting this to true tells Xalan to provide a JAXP SourceLocator a program can use to determine the location (line numbers, column numbers, system IDs, and public IDs) of individual nodes during the transform. However, it engenders a substantial performance hit so it’s turned off by default.

Other processors define their own attributes. Although TrAX is designed as a generic API, it does let you access such custom features with these two methods:

public abstract void setAttribute(String name, Object value)
    throws IllegalArgumentException;
public abstract Object getAttribute(String name)
    throws IllegalArgumentException;

For example, this code tries to turn on incremental output:

TransformerFactory xformFactory
 = TransformerFactory.newInstance();
try {
  xformFactory.setAttribute(
   "http://apache.org/xalan/features/incremental", Boolean.TRUE);
}
catch (IllegalArgumentException e) {
  // This XSLT processor does not support the
  // http://apache.org/xalan/features/incremental attribute,
  // but we can still use the processor anyway
}

If you’re using any processor except Xalan-J 2.x. this will not exactly fail but it won’t exactly succeed either. Using non-standard attributes may limit the portability of your programs. However most attributes (and all of the Xalan attributes) merely adjust how the processor achieves its result. They do not change the final result in any way.

URI Resolution

An XSLT stylesheet can use the document() function to load additional source documents for processing. It can also import or include additional stylesheets with the xsl:import and xsl:include instructions. In all three cases the document to load is identified by a URI.

Normally a Transformer simply loads the document at that URL. However, you can redirect the request to a proxy server, to local copies, or to previously cached copies using a URIResolver. This interface, summarized in Example 17.8, returns Source objects for a specified URL and an optional base. It is similar in intent to SAX’s EntityResolver. However, EntityResolver is based on public and system IDs whereas this interface is based on URLs and base URLs.

Example 17.8. The TrAX URIResolver interface

package javax.xml.transform;

public interface URIResolver {

  public Source resolve(String href, String base)
   throws TransformerException;

}

The resolve() method should return a Source object if it successfully resolves the URL. Otherwise it should return null to indicate that the default URL resolution mechanism should be used. For example, Example 17.9 is a simple URIResolver implementation that looks for a gzipped version of a document (i.e. one that ends in .gz). If it finds one it uses the java.util.zip.GZIPInputStream class to build a StreamSource from the gzipped document. Otherwise, it returns null and the usual methods for resolving URLs are followed.

Example 17.9. A URIResolver class

import javax.xml.transform.URIResolver;
import javax.xml.transform.stream.StreamSource;
import java.util.zip.GZIPInputStream;
import java.net.URL;
import java.io.InputStream;

public class GZipURIResolver implements URIResolver {

  public Source resolve(String href, String base) {

    try {
      href = href + ".gz";
      URL context = new URL(base);
      URL u = new URL(context, href);
      InputStream in = u.openStream();
      GZIPInputStream gin = new GZIPInputStream(in);
      return new StreamSource(gin, u.toString());
    }
    catch (Exception e) {
      // If anything goes wrong, just return null and let
      // the default resolver try.
    }
    return null;
  }

}

The following two methods in TransformerFactory set and get the URIResolver that Transformer objects created by this factory will use to resolve URIs:

public abstract void setURIResolver(URIResolver resolver);
public abstract URIResolver getURIResolver();

For example,

URIResolver resolver = new GZipURIResolver();
factory.setURIResolver(resolver);

Error Handling

XSLT transformations can fail for any of several reasons, including:

  • The stylesheet is syntactically incorrect.

  • The source document is malformed.

  • Some external resource the processor needs to load, such as a document referenced by the document() function or the .class file that implements an extension function, is not available.

By default, any such problems are reported by printing them on System.err. However, you can provide more sophisticated error handling, reporting, and logging by implementing the ErrorListener interface. This interface, shown in Example 17.10, is modeled after SAX’s ErrorHandler interface. Indeed aside from the fact that the arguments are all TransformerExceptions instead of SAXExceptions, it’s almost identical.

Example 17.10. The TrAX ErrorListener interface

package javax.xml.transform;

public interface ErrorListener {

  public void warning(TransformerException exception)
   throws TransformerException;
  public void error(TransformerException exception)
   throws TransformerException;
  public void fatalError(TransformerException exception)
   throws TransformerException;

}

Example 17.11 demonstrates with a simple class that uses the java.util.logging package introduced in Java 1.4 to report errors rather than printing them on System.err. Each exception is logged to a Logger specified in the constructor. Unfortunately the Logging API doesn’t really have separate categories for fatal and non-fatal errors so I just classify them both as “severe”. (You could define a custom subclass of Level that did differentiate fatal and non-fatal errors; but since this is not a book about the Logging API, I leave that as exercise for the reader.)

Example 17.11. An ErrorListener that uses the Logging API

import javax.xml.transform.*;
import java.util.logging.*;

public class LoggingErrorListener implements ErrorListener {

  private Logger logger;

  public LoggingErrorListener(Logger logger) {
    this.logger = logger;
  }

  public void warning(TransformerException exception) {

    logger.log(Level.WARNING, exception.getMessage(), exception);

    // Don't throw an exception and stop the processor
    // just for a warning; but do log the problem
  }

  public void error(TransformerException exception)
   throws TransformerException {

    logger.log(Level.SEVERE, exception.getMessage(), exception);
    // XSLT is not as draconian as XML. There are numerous errors
    // which the processor may but does not have to recover from;
    // e.g. multiple templates that match a node with the same
    // priority. I do not want to allow that so I throw this
    // exception here.
    throw exception;

  }

  public void fatalError(TransformerException exception)
   throws TransformerException {

    logger.log(Level.SEVERE, exception.getMessage(), exception);

    // This is an error which the processor cannot recover from;
    // e.g. a malformed stylesheet or input document
    // so I must throw this exception here.
    throw exception;

  }

}

The following two methods appear in both TransformerFactory and Transformer. They enable you to set and get the ErrorListener that the object will report problems to:

public abstract void setErrorListener(ErrorListener listener)
    throws IllegalArgumentException;
public abstract ErrorListener getErrorListener();

An ErrorListener registered with a Transformer will report errors with the transformation. An ErrorListener registered with a TransformerFactory will report errors with the factory’s attempts to create new Transformer objects. For example, this code fragment installs separate LoggingErrorListeners on the TransformerFactory and the Transformer object it creates that will record messages in two different logs.

TransformerFactory factory = TransformerFactory.newInstance();
Logger factoryLogger
 = Logger.getLogger("com.macfaq.trax.factory");
ErrorListener factoryListener
 = new LoggingErrorListener(factoryLogger);
factory.setErrorListener(factoryListener);
Source source = new StreamSource("FibonacciXMLRPC.xsl");
Transformer stylesheet = factory.newTransformer(source);
Logger transformerLogger
 = Logger.getLogger("com.macfaq.trax.transformer");
ErrorListener transformerListener
 = new LoggingErrorListener(transformerLogger);
stylesheet.setErrorListener(transformerListener);

Passing Parameters to Style Sheets

Top-level xsl:param and xsl:variable elements both define variables by binding a name to a value. This variable can be dereferenced elsewhere in the stylesheet using the form $name. Once set, the value of an XSLT variable is fixed and cannot be changed. However if the variable is defined with a top-level xsl:param element instead of an xsl:variable element, then the default value can be changed before the transformation begins.

For example, the DocBook XSL stylesheets I use to generate this book have a number of parameters that set various formatting options. For this book I use these settings:

  <xsl:param name="fop.extensions">1</xsl:param>
  <xsl:param name="page.width.portrait">7.375in</xsl:param>
  <xsl:param name="page.height.portrait">9.25in</xsl:param>
  <xsl:param name="page.margin.top">0.5in</xsl:param>
  <xsl:param name="page.margin.bottom">0.5in</xsl:param>
  <xsl:param name="region.before.extent">0.5in</xsl:param>
  <xsl:param name="body.margin.top">0.5in</xsl:param>
  <xsl:param name="page.margin.outer">1.0in</xsl:param>
  <xsl:param name="page.margin.inner">1.0in</xsl:param>
  <xsl:param name="body.font.family">Times</xsl:param>
  <xsl:param name="variablelist.as.blocks" select="1"/>
  <xsl:param name="generate.section.toc.level" select="1"/>
  <xsl:param name="generate.component.toc" select="0"/>

The initial (and thus final) value of any parameter can be changed inside your Java code using these three methods of the Transformer class:

public abstract void setParameter(String name, Object value);
public abstract Object getParameter(String name);
public abstract void clearParameters();

The setParameter() method provides a value for a parameter that overrides any value used in the stylesheet itself. The processor is responsible for converting the Java object type passed to a reasonable XSLT equivalent. This should work well enough for String, Integer, Double, and Boolean as well as DOM types like Node and NodeList. However, I wouldn’t rely on it for anything more complex like a File or a Frame.

The getParameter() method returns the value of a parameter previously set by Java. It will not return any value from the stylesheet itself, even if it has not been overridden by the Java code. Finally, the clearParameters() method eliminates all Java mappings of parameters so that those variables are returned to whatever value is specified in the stylesheet.

For example, in Java the above list of parameters for the DocBook stylesheets could be set with a JAXP Transformer object like this:

transformer.setParameter("fop.extensions", "1");
transformer.setParameter("page.width.portrait", "7.375in");
transformer.setParameter("page.height.portrait", "9.25in");
transformer.setParameter("page.margin.top", "0.5in");
transformer.setParameter("region.before.extent", "0.5in");
transformer.setParameter("body.margin.top", "0.5in");
transformer.setParameter("page.margin.bottom", "0.5in");
transformer.setParameter("page.margin.outer", "1.0in");
transformer.setParameter("page.margin.inner", "1.0in");
transformer.setParameter("body.font.family", "Times");
transformer.setParameter("variablelist.as.blocks", "1");
transformer.setParameter("generate.section.toc.level", "1");
transformer.setParameter("generate.component.toc", "0");

Here I used strings for all the values. However, in a few cases I could have used a Number of some kind instead.

Output Properties

XSLT is defined in terms of a transformation from one tree to a different tree, all of which takes place in memory. The actual conversion of that tree to a stream of bytes or a file is an optional step. If that step is taken, the xsl:output instruction controls the details of serialization. For example, it can specify XML, HTML, or plain text output. It can specify the encoding of the output, what the document type declaration points to, whether the elements should be indented, what the value of the standalone declaration is, where CDATA sections should be used, and more. For example, adding this xsl:output element to a stylesheet would produce plain text output instead of XML:

<xsl:output
  method="text"
  encoding="US-ASCII"
  media-type="text/plain"
/>

This xsl:output element asks for pretty-printed XML:

<xsl:output
  method="xml"
  encoding="UTF-16"
  indent="yes"
  media-type="text/xml"
  standalone="yes"
/>

In all, there are ten attributes of the xsl:output element that control serialization of the result tree:

method="xml | html | text"

The output method. xml is the default. html uses classic HTML syntax such as <hr> instead of <hr />. text outputs plain text but no markup.

version="1.0"

The version number used in the XML declaration. Currently, this should always have the value 1.0.

encoding="UTF-8 | UTF-16 | ISO-8859-1 | …"

The encoding used for the output and in the encoding declaration of the output document.

omit-xml-declaration="yes | no"

yes if the XML declaration should be omitted, no otherwise. (i.e. no if the XML declaration should be included, yes if it shouldn’t be.) The default is no.

standalone="yes | no"

The value of the standalone attribute for the XML declaration; either yes or no

doctype-public="public ID"

The public identifier used in the DOCTYPE declaration

doctype-system="URI"

The URL used as a system identifier in the DOCTYPE declaration

cdata-section-elements="element_name_1 element_name_2 …"

A white space separated list of the qualified names of the elements’ whose content should be output as a CDATA section

indent="yes | no"

yes if extra white space should be added to pretty-print the result, no otherwise. The default is no.

media-type="text/xml | text/html | text/plain | application/xml… "

The MIME media type of the output such as text/html, application/xml, or application/xml+svg

Note

All of these output properties are at the discretion of the XSLT processor. The processor is not required to serialize the result tree at all, much less to serialize it with extra white space, a document type declaration, and so forth. In particular, I have encountered XSLT processors that only partially support indent="yes".

You can also control these output properties from inside your Java programs using these four methods in the Transformer class. You can either set them one by one or as a group with the java.util.Properties class.

public abstract void setOutputProperties(Properties outputFormat)
    throws IllegalArgumentException;
public abstract Properties getOutputProperties();
public abstract void setOutputProperty(String name, String value)
    throws IllegalArgumentException;
public abstract String getOutputProperty(String name);

The keys and values for these properties are simply the string names established by the XSLT 1.0 specification. For convenience, the javax.xml.transform.OutputKeys class shown in Example 17.12 provides named constants for all the property names.

Example 17.12. The TrAX OutputKeys class

package javax.xml.transform;

public class OutputKeys {

  private OutputKeys() {}

  public static final String METHOD = "method";
  public static final String VERSION = "version";
  public static final String ENCODING = "encoding";
  public static final String OMIT_XML_DECLARATION
   = "omit-xml-declaration";
  public static final String STANDALONE = "standalone";
  public static final String DOCTYPE_PUBLIC = "doctype-public";
  public static final String DOCTYPE_SYSTEM = "doctype-system";
  public static final String CDATA_SECTION_ELEMENTS
   = "cdata-section-elements";
  public static final String INDENT = "indent";
  public static final String MEDIA_TYPE = "media-type";

}

For example, this Java code fragment has the same effect as the above xsl:output element:

transformer.setOutputProperty(OutputKeys.METHOD, "xml");
transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-16");
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
transformer.setOutputProperty(OutputKeys.MEDIA_TYPE, "text/xml");
transformer.setOutputProperty(OutputKeys.STANDALONE, "yes");

In the event of a conflict between what the Java code requests with output properties requests and what the stylesheet requests with an xsl:output element, the ones specified in the Java code take precedence.

Sources and Results

The Source and Result interfaces abstract out the API dependent details of exactly how an XML document is represented. You can construct sources from DOM nodes, SAX event sequences, and raw streams. You can target the result of a transform at a DOM Node, a SAX ContentHandler, or a stream-based target such as an OutputStream, Writer, File, or String. Other models may also provide their own implementations of these interfaces. For instance, JDOM has an org.jdom.transform package that includes a JDOMSource and JDOMResult class.

In fact, these different models have very little in common, other than that they all hold an XML document. Consequently, the Source and Result interfaces don’t themselves provide a lot of the functionality you need, just methods to get the system and public ID of the document. Everything else is deferred to the implementations. In fact, XSLT engines generally need to work directly with the subclasses rather than with the generic superclasses; and not all engines are able to process all three kinds of sources and targets. Polymorphism just doesn’t work very well here.

Note

It is important to set at least the system IDs of your sources because some parts of the stylesheet may rely on this. In particular, if any of your xsl:import or xsl:include elements or document() functions contain relative URLs, then they’ll be resolved relative to the URL of the stylesheet source.

DOMSource and DOMResult

A DOMSource is a wrapper around a DOM Node. The DOMSource class provides methods to set and get the node that serves as the root of the transform, as well as the system and public IDs of that node.

Example 17.13. The TrAX DOMSource class

package javax.xml.transform.dom;

public class DOMSource implements Source {

  public static final String FEATURE =
    "http://javax.xml.transform.dom.DOMSource/feature";

  public DOMSource() {}
  public DOMSource(Node node);
  public DOMSource(Node node, String systemID);

  public void    setNode(Node node);
  public Node   getNode();
  public void    setSystemId(String baseID);
  public String getSystemId();

}

In theory, you should be able to convert any DOM Node object into a DOMSource and transform it. In practice, only transforming document nodes is truly reliable. (It’s not even clear that the XSLT processing model applies to anything that isn’t a complete document.) In my tests, Xalan-J could transform all the nodes I threw at it. However, Saxon could only transform Document objects and Element objects that were part of a document tree.

A DOMResult is a wrapper around a DOM Document, DocumentFragment, or Element Node to which the output of the transform will be appended. The DOMResult class provides constructors and methods to set and get the node that serves as the root of the transform, as well as the system and public IDs of that node.

Example 17.14. The TrAX DOMResult class

package javax.xml.transform.dom;

public class DOMResult implements Result {

  public static final String FEATURE =
  "http://javax.xml.transform.dom.DOMResult/feature";

  public DOMResult();
  public DOMResult(Node node);
  public DOMResult(Node node, String systemID);

  public void setNode(Node node);
  public Node getNode();
  public void setSystemId(String systemId);
  public String getSystemId();

}

If you specify a Node for the result, either via the constructor or by calling setNode(), then the output of the transform will be appended to that node’s children. Otherwise, the transform output will be appended to a new Document or DocumentFragment Node. The getNode() method returns this Node.

SAXSource and SAXResult

The SAXSource class, shown in Example 17.15, provides input to the XSLT processor read from a SAX InputSource by an XMLReader.

Example 17.15. The TrAX SAXSource class

package javax.xml.transform.sax;

public class SAXSource implements Source {

  public static final String FEATURE =
   "http://javax.xml.transform.sax.SAXSource/feature";

  public SAXSource();
  public SAXSource(XMLReader reader, InputSource inputSource);
  public SAXSource(InputSource inputSource);

  public void        setXMLReader(XMLReader reader);
  public XMLReader   getXMLReader();
  public void        setInputSource(InputSource inputSource);
  public InputSource getInputSource();
  public void        setSystemId(String systemID);
  public String      getSystemId();

  public static InputSource sourceToInputSource(Source source);

}

The SAXResult class, shown in Example 17.16, receives output from the XSLT processor as a stream of SAX events fired at a specified ContentHandler and optional LexicalHandler.

Example 17.16. The TrAX SAXResult class

package javax.xml.transform.sax;

public class SAXResult implements Result

  public static final String FEATURE =
   "http://javax.xml.transform.sax.SAXResult/feature";

  public SAXResult();
  public SAXResult(ContentHandler handler);

  public void           setHandler(ContentHandler handler);
  public ContentHandler getHandler();
  public void           setLexicalHandler(LexicalHandler handler);
  public LexicalHandler getLexicalHandler();
  public void           setSystemId(String systemId);
  public String         getSystemId();

}

StreamSource and StreamResult

The StreamSource and StreamResult classes are used as sources and targets for transforms from sequences of bytes and characters. This includes streams, readers, writers, strings, and files. What unifies these is that none of them know they contain an XML document. Indeed, on input they may not always contain an XML document. If so, an exception will be thrown as soon as you attempt to build a Transformer or a Templates object from the StreamSource.

The StreamSource class, shown in Example 17.17, provides constructors and methods to get and set the actual source of data.

Example 17.17. The TrAX StreamSource class

package javax.xml.transform.stream;

public class StreamSource implements Source {

  public static final String FEATURE =
   "http://javax.xml.transform.stream.StreamSource/feature";

  public StreamSource();
  public StreamSource(InputStream inputStream);
  public StreamSource(InputStream inputStream, String systemID);
  public StreamSource(Reader reader);
  public StreamSource(Reader reader, String systemID);
  public StreamSource(String systemID);
  public StreamSource(File f);

  public void        setInputStream(InputStream inputStream);
  public InputStream getInputStream();
  public void        setReader(Reader reader);
  public Reader      getReader();
  public void        setPublicId(String publicID);
  public String      getPublicId();
  public void        setSystemId(String systemID);
  public String      getSystemId();
  public void        setSystemId(File f);

}

You should not specify both an InputStream and a Reader. If you do, which one the processor reads from is implementation dependent. If neither an InputStream nor a Reader is available, then the processor will attempt to open a connection to the URI specified by the system ID. You should set the system ID even if you do specify an InputStream or a Reader because this will be needed to resolve relative URLs that appear inside the stylesheet and input document.

The StreamResult class, shown in Example 17.18, provides constructors and methods to get and set the actual target of the data.

Example 17.18. The TrAX StreamResult class

package javax.xml.transform.stream;

public class StreamResult implements Result

  public static final String FEATURE =
   "http://javax.xml.transform.stream.StreamResult/feature";

  public StreamResult() {}
  public StreamResult(OutputStream outputStream);
  public StreamResult(Writer writer);
  public StreamResult(String systemID);
  public StreamResult(File f);

  public void         setOutputStream(OutputStream outputStream);
  public OutputStream getOutputStream();
  public void         setWriter(Writer writer);
  public Writer       getWriter();
  public void         setSystemId(String systemID);
  public void         setSystemId(File f);
  public String       getSystemId();

}

You should specify the system ID URL and one of the other identifiers (File, OutputStream, Writer, or String.) If you specify more than one possible target, which one the processor chooses is implementation dependent.

Prev | Up | Next
Chapter 17. XSLT | Home | Extending XSLT with Java
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 19, 2002


Extending XSLT with Java
Prev Chapter 17. XSLT Next

Extending XSLT with Java

TrAX lets you integrate XSLT code with Java programs. Most XSLT processors written in Java also let you go the other way, integrating Java code with XSLT stylesheets. The most common reason to do this is to provide access to operating system functionality XSLT doesn’t offer such as querying a database, listing the files in a directory, or asking the user for more information with a dialog box. Java can also be used when you simply find it easier to implement some complex algorithm in imperative Java rather than functional XSLT. For example, although you can do complicated string searching and replacing in XSLT, I guarantee you it will be about a thousand times easier in Java, especially with a good regular expression library. And finally, even though a function could be implemented in pure XSLT relatively easily, you may choose to write it in Java anyway purely for performance reasons. This is especially true for mathematical functions like factorial and fibonacci. XSLT optimizers are not nearly as mature or as reliable as Java optimizers, and those that do exist mostly focus on optimizing XPath search and evaluation on node-sets rather than mathematical operations on numbers.

XSLT defines two mechanisms for integrating Java code into stylesheets, extension functions and extension elements. These are invoked exactly like built-in functions and elements such as document() and xsl:template. However, rather than being provided by the processor, they’re written in Java. Furthermore, they have names in some non-XSLT namespace. The exact way such functions and elements are linked with the processor varies from processor to processor though.

Regardless of which XSLT processor you’re using, there are two basic parts to writing and using extension functions and elements:

  • Binding the extensions to the stylesheet. This is done via namespaces, class names, and the Java class path.

  • Mapping the five XSLT types (number, boolean, string, node-set, and result tree fragment) to Java types and vice versa.

Extension Functions

As an example, I’m going to write a simple extension function that calculates Fibonacci numbers. This can be used as a faster alternative to the earlier recursive template. Example 17.19 contains this function. The entire class is in the com.macfaq.math package. When writing extension functions and elements, you really have to use proper Java package naming and set up your class path appropriately.

Example 17.19. A Java class that calculates Fibonacci numbers

package com.macfaq.math;

import java.math.BigInteger;

public class FibonacciNumber {

  public static BigInteger calculate(int n) {

    if (n <= 0) {
      throw new IllegalArgumentException(
       "Fibonacci numbers are only defined for positive integers"
      );
    }
    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;

    for (int i = 3; i <= n; i++) {
      BigInteger temp = high;
      high = high.add(low);
      low = temp;
    }

    return high;

  }

}

Notice there’s nothing about XSLT in this example. This is just like any other Java class. On the Java side, all you need to do to make it accessible to the XSLT processor is compile it and install the .class file in the proper place in the processor’s class path.

If the extension function throws an exception, as calculate() might if it’s passed a negative number as an argument, then the XSLT processing will halt. XSLT has no way to catch and respond to exceptions thrown by extension functions. Consequently, if you want to handle them, you’ll need to handle them in the Java code. After catching the exception, you’ll want to return something. Possibilities include:

  • A String containing an error message

  • A NodeList containing a fault document

  • An integer error code

Since this may not be the same type you normally return, you’ll probably need to declare that the method returns Object to give you the additional flexibility. For example, this method returns an error message inside a String instead of throwing an exception:

  public static Object calculate(int n) {

    if (n <= 0) {
     return
      "Fibonacci numbers are only defined for positive integers";
    }
    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;

    for (int i = 3; i <= n; i++) {
      BigInteger temp = high;
      high = high.add(low);
      low = temp;
    }

    return high;

  }

This method returns -1 (an illegal value for a Fibonacci number) instead of throwing an exception:

  public static BigInteger calculate(int n) {

    if (n <= 0) return new BigInteger("-1");
    BigInteger low  = BigInteger.ONE;
    BigInteger high = BigInteger.ONE;

    for (int i = 3; i <= n; i++) {
      BigInteger temp = high;
      high = high.add(low);
      low = temp;
    }

    return high;

  }

It would be up to the stylesheet to check for the error code before using the result, and handle such a situation appropriately. In this example, that might require calling the extension function before any output is generated, storing the result in a variable, and deciding whether to output a successful response or a fault document based on the value of that variable. Waiting until the template for the int element is activated would be too late because by that point substantial parts of a successful response document have already been generated.

Now we need a stylesheet that uses this function to calculate Fibonacci numbers instead of the XSLT template. The details at this point are a little processor specific. I will cover the two most popular, Saxon and Xalan. As you’ll see there are quite a few points of similarity between them (though I think Saxon’s approach is the cleaner of the two). Most other processors are likely to use something similar.

Tip

Before spending a lot of time and effort writing your own extension functions, check to see if the EXSLT library already has the extension function you need. EXSLT provides many useful extension functions and elements for working with dates and times, functions, math, strings, regular expressions, sets, and more. This library has been ported to many different processors in many different platforms and languages. I use some of the date functions in the stylesheets for this book.

Extension functions in Saxon

Saxon allows you to bind any Java class to a namespace prefix. The trick is to use the custom URI scheme java followed by a colon and the fully package-qualified name of the class. For example, this attribute binds the namespace prefix fib to the com.macfaq.math.FibonacciNumber class:

xmlns:fib="java:com.macfaq.math.FibonacciNumber"

As long as this mapping is in scope, you can invoke any static function in the com.macfaq.math.FibonacciNumber class by using the prefix fib and the name of the method. For example, the old template for the int element could be replaced by this one:

  <xsl:template match="int"
                xmlns:fib="java:com.macfaq.math.FibonacciNumber">
    <int>
      <xsl:value-of select="fib:calculate(number(.))"/>
    </int>
  </xsl:template>

Here the number() function converts the value of the context node to an XSLT number. Then the processor looks for a static method named calculate() in the Java class mapped to the fib prefix that takes a single argument. It finds one, invokes it, and inserts the return value into the result tree.

XSLT is much more weakly typed than Java, and this can be useful when writing extension functions. Saxon will only invoke methods that have the right name and the right number of arguments. However, it will often convert the types of arguments and return values as necessary to make a function fit. In this case, the calculate() method expects to receive an int. However, an XSLT number is really more like a Java double. In this case, since Saxon can’t find a matching method that takes a double it truncates the fractional part of the double to get an int and invokes the method that takes an int. This is a conversion that Java itself would not do without an explicit cast.

Going in the opposite direction, the calculate() method returns a BigInteger, which is not equivalent to any of XSLT’s types. Thus Saxon converts it to a string using its toString() before inserting it into the result tree. Other more recognizable return types may be converted differently. For example, void is converted to an empty node-set and primitive number types like int and double are converted to XSLT numbers as are type-wrapper classes like Integer and Double. A DOM NodeList is converted to an XPath node-set. However, the nodes in the list must all be created by Saxon’s own DOM implementation. You can’t use third party DOM implementations like Xerces or GNU JAXP in a Saxon extension function.

Tip

Namespace mappings for extension functions and elements are normally only relevant in the stylesheet. Nonetheless they often have an annoying habit of popping up in the output document. If you know that an extension element or function prefix will not be used in the output document (and 99% of the time you do know exactly this) you can add an exclude-result-prefixes attribute to the stylesheet root element that contains a list of the namespace prefixes whose declarations should not be copied into the output document. For example,

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:fib="java:com.macfaq.math.FibonacciNumber"
  xmlns:saxon="http://icl.com/saxon"
  exclude-result-prefixes="fib saxon">
Instance Methods and Constructors

XSLT is not an object oriented language. Static methods fit much more neatly into its structures than do objects and instance methods. If I’m writing a method just for XSLT, I’ll normally make it static if at all possible. However, Saxon can use instance methods as extension functions too. As before, the fully package qualified class name must be bound to a namespace prefix. The constructor for the class can be called using the special local function name new(). For example, this template retrieves the current time using the Java Date class:

<xsl:template name="currentTime"
              xmlns:date="java:java.util.Date">
  <xsl:value-of select="date:new()"/>
</xsl:template>

date:new() in XSLT is basically the same thing as new Date() in Java. When the Date constructor is invoked with no arguments, Java initializes the resulting Date object to the current time. You can also pass arguments to constructors, just like you can to static methods.

The object the new() function returns is normally assigned to a variable. You can pass this variable to other extension functions as an argument. To invoke instance methods on that object, pass the variable that points to the object whose instance method you’re invoking as the first argument to the instance method. Then the normal first argument gets pushed over to become the second argument, the second argument becomes the third, and so on. For example, this template uses the GregorianCalendar class to get today’s date. First it uses the static getInstance() method to return a GregorianCalendar object initialized to the current time. Then it passes the appropriate integer constants to the get() instance method to retrieve the month, day, and year. It produces the current date in the form 2002-3-26.

<xsl:template name="today"
              xmlns:cal="java:java.util.GregorianCalendar">
  <xsl:variable name="rightNow" select="cal:getInstance()" />
  <!-- The Calendar class uses zero-based months;
       i.e. January is month 0, February is month 1, and
       so on. We have to add one to get the customary month
       number. -->
  <xsl:variable name="month" select="cal:get($rightNow, 2) + 1" />
  <xsl:variable name="day" select="cal:get($rightNow, 5)" />
  <xsl:variable name="year" select="cal:get($rightNow, 1)" />
  <xsl:value-of
   select="$year" />-<xsl:value-of
   select="$month" />-<xsl:value-of
   select="$day" />
</xsl:template>

Note

If I were writing this in Java rather than XSLT, the code would look like this:

Calendar rightNow = Calendar.getInstance();
// Months are zero-based; i.e. January is month 0, February is
// month 1, and so on. We have to add one to get the customary
// month number.
String month = rightNow.get(Calendar.MONTH) + 1;
String date  = rightNow.get(Calendar.DATE);
String year  = rightNow.get(Calendar.YEAR);
String result = year + "-" + month + "-" + date;

However, Saxon doesn’t support extension fields so XSLT must use the actual constant key values instead of the named constants.

If you absolutely have to use the value of a field, (e.g. because a method expects an instance of the type-safe enum pattern instead of an int constant), you can always write an extension function whose sole purpose is to return the relevant field.

Extension functions in Xalan

Xalan’s extension function mechanism is a little more complicated and a little more powerful than Saxon’s, but not a great deal more. Xalan offers somewhat greater access to the XSLT context inside extension functions if you need it, and has some additional shortcuts for mapping Java classes to namespace prefixes. Most importantly, it allows extension functions to work with any compliant DOM2 implementation, rather than requiring its own custom DOM.

Xalan uses the custom URI scheme xalan to bind namespace prefixes to classes. To bind a Java class to a namespace prefix in Xalan, you add an attribute of the form xmlns:prefix="xalan://packagename.classname" to the root element of the stylesheet or some other ancestor element. For example, this attribute binds the namespace prefix fib to the com.macfaq.math.FibonacciNumber class:

xmlns:fib="xalan://com.macfaq.math.FibonacciNumber"

As long as this mapping is in scope, you can invoke any static function in the com.macfaq.math.FibonacciNumber class by using the prefix fib and the name of the method. For example, the pure XSLT template for the int element could be replaced by this one:

<xsl:template match="int"
   xmlns:fib="xalan://com.macfaq.math.FibonacciNumber">
  <int>
    <xsl:value-of select="fib:calculate(number(.))"/>
  </int>
</xsl:template>

Xalan also allows you to define a namespace prefix for the entire Java class library by associating it with the URI http://xml.apache.org/xslt/java. The function calls must then use fully qualified class names. For example, this template uses the prefix java to identify extension functions:

<xsl:template match="int"
    xmlns:java="http://xml.apache.org/xslt/java">
  <int>
    <xsl:value-of select=
     "java:com.macfaq.math.FibonacciNumber.calculate(number(.))"
    />
  </int>
</xsl:template>

This form is convenient if your stylesheets use many different classes. It is of course not limited to classes you write yourself. It works equally well for classes from the standard library and third-party libraries. For example, here’s a random template that uses Java’s Math.random() method:

<xsl:template name="random"
              xmlns:java="http://xml.apache.org/xslt/java">
  <xsl:value-of select="java:java.lang.Math.random()" />
</xsl:template>
Constructors and Instance Methods

Xalan can use instance methods as extension functions too. The new() function invokes the constructor for the class and can take whatever arguments the constructor requires. For example, this template retrieves the current time using the Java Date class:

<xsl:template name="currentTime"
              xmlns:java="http://xml.apache.org/xslt/java">
  <xsl:value-of select="java:java.util.Date.new()"/>
</xsl:template>

If the prefix is bound to a specific class, you can omit the class name. For example,

<xsl:template name="currentTime"
              xmlns:date="xalan://java.util.Date">
  <xsl:value-of select="date:new()"/>
</xsl:template>

The object the new() function returns can be assigned to an XSLT variable that can then be passed as an argument to other extension functions or used to invoke instance methods on the object. As in Saxon, to invoke an instance method pass the object whose method you’re invoking as the first argument to the method. For example, here’s the Xalan version of the GregorianCalendar template that produces the current date in the form 2002-3-26.

<xsl:template name="today"
              xmlns:cal="xalan://java.util.GregorianCalendar">
  <xsl:variable name="rightNow" select="cal:getInstance()" />
  <!-- The GregorianCalendar class counts months from zero
       so we have to add one to get the customary number -->
  <xsl:variable name="month" select="cal:get($rightNow, 2) + 1" />
  <xsl:variable name="day" select="cal:get($rightNow, 5) " />
  <xsl:variable name="year" select="cal:get($rightNow, 1)" />
  <xsl:value-of
   select="$year" />-<xsl:value-of
   select="$month" />-<xsl:value-of
   select="$day" />
</xsl:template>

Like Saxon, Xalan also doesn’t let you access fields in a class, so once again it’s necessary to use the actual values instead of the named constants for the arguments to the get() method.

Exceptions thrown by extension functions have the same results in Xalan as in Saxon; that is, the the XSLT processing halts, possibly in the middle of transforming a document. Once again, it’s probably a good idea to design your extension functions so that they handle all probable exceptions internally and always return a sensible result.

Type Conversion

Xalan converts method arguments and return types between Java and XSLT types in a mostly intuitive way. Table 17.1 lists the conversions from XSLT’s five types to Java types in order of preference:

Table 17.1. Xalan Conversions from XSLT to Java

XSLT type | Java types (in decreasing order of preference)
node-set | org.w3c.dom.traversal.NodeIterator, org.w3c.dom.NodeList, org.w3c.dom.Node, String, Object, char, double, float, long, int, short, byte, boolean
string | String, Object, char, double, float, long, int, short, byte, boolean
boolean | boolean, Boolean, Object, String
number | double, Double, float, long, int, short, char, byte, boolean, String, Object
result tree fragment | org.w3c.dom.traversal.NodeIterator, org.w3c.dom.NodeList, org.w3c.dom.Node, String, Object, char, double, float, long, int, short, byte, boolean

Moving in the other direction from Java to XSLT, the conversions are fairly obvious. Table 17.2 summarizes them. Besides the ones listed here, other object types will normally be converted to a string using their toString() method if they’re actually dereferenced somewhere in the stylesheet. However, their original type will be maintained when they’re passed back to another extension function.

Table 17.2. Xalan Conversions from Java to XSLT

Java type | Xalan XSLT type
org.w3c.dom.traversal.NodeIterator | node-set
org.apache.xml.dtm.DTM | node-set
org.apache.xml.dtm.DTMAxisIterator | node-set
org.apache.xml.dtm.DTMIterator | node-set
org.w3c.dom.Node and its subtypes (Element, Attr, etc) | node-set
org.w3c.dom.DocumentFragment | result tree fragment
String | string
Boolean | boolean
Number and its subclasses (Double, Integer, etc) | number
double | number
float | number
int | number
long | number
short | number
byte | number
char | object
boolean | boolean
null | empty string
void | empty string
Expression Context

There is one thing Xalan extension functions can do that Saxon extension functions can’t. A Xalan extension function can receive the current XSLT context as an argument. This provides information about the context node, the context node position, the context node list, and variable bindings. Admittedly, needing to know this information inside an extension function is rare. Most operations that consider the current context are more easily implemented in XSLT than Java. Nonetheless, if you need to know this for some reason, you can declare that the initial argument to your function has type org.apache.xalan.extensions.ExpressionContext; for example,

public static Node findMaximum(ExpressionContext context);

You do not need to pass an argument of this type explicitly. Xalan will create an ExpressionContext object for you and pass it to the method automatically. Furthermore, Xalan will always pick a method that takes an ExpressionContext over one that does not.

This Xalan-J ExpressionContext interface, shown in Example 17.20, provides methods to get the context and the context node list, convert the context node into either its string or number value (as defined by the XPath string() and number() functions), and to get the XPath object bound to a known variable or parameter.

Example 17.20. The Xalan ExpressionContext interface

package org.apache.xalan.extensions;

public interface ExpressionContext {

  public Node         getContextNode();
  public NodeIterator getContextNodes();
  public double       toNumber(Node n);
  public String       toString(Node n);
  public XObject      getVariableOrParam(
   org.apache.xml.utils.QName qualifiedName)
   throws javax.xml.transform.TransformerException;

}

Extension Elements

An extension element is much like an extension function. However in the stylesheet it appears as an entire element such as <saxon:script/> or <redirect:write /> rather than as a mere function in an XPath expression contained in a select or test attribute. Any value it returns is placed directly in the result tree.

For example, suppose you wanted to define a fibonacci element like this one:

<fib:fibonacci xmlns:fib="java:com.macfaq.math.FibonacciNumber">
  10
</fib:fibonacci>

When processed, this element would be replaced by the specified Fibonacci number.

The first question is how the XSLT processor should recognize this as an extension element. After all, fib:fibonacci looks just like a literal result element that should be copied verbatim. The answer is that the xsl:stylesheet root element (or some other ancestor element) should have an extension-element-prefixes attribute containing a whitespace separated list of namespace prefixes that identify extension elements. For example, this stylesheet uses the saxon and fib prefixes for extension elements:

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:saxon="http://icl.com/saxon"
  xmlns:fib="java:com.macfaq.math.FibonacciNumber"
  extension-element-prefixes="saxon fib">

  <!- - ... - ->

</xsl:stylesheet>

Since you can’t be sure which extension elements are likely to be available across processors, it’s customary to include one or more xsl:fallback elements as children of each extension element. Each such element contains a template that is instantiated if and only if the parent extension element can’t be found. Example 17.21 demonstrates a stylesheet that attempts to use the fib:fibonacci extension element. However, if that element cannot be found then a pure XSLT solution is used instead.

Example 17.21. A stylesheet that uses an extension element

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:fib="http://namespaces.cafeconleche.org/fibonacci"
  extension-element-prefixes="fib">

  <!-- I deleted the validation code from this stylesheet to
       save space, but it would be easy to add back in if
       for production use. -->

  <xsl:template match="/methodCall">
    <methodResponse>
      <params>
        <param>
          <value>
            <xsl:apply-templates select="params/param/value" />
          </value>
        </param>
      </params>
    </methodResponse>
  </xsl:template>

  <xsl:template match="value">
    <int>
      <fib:fibonacci>
        <xsl:value-of select="number(.)"/>
        <xsl:fallback>
          <!-- This template will be called only if the
               fib:fibonacci code can't be loaded. -->
          <xsl:call-template name="calculateFibonacci">
            <xsl:with-param name="index" select="number(.)" />
          </xsl:call-template>
        </xsl:fallback>
      </fib:fibonacci>
    </int>
  </xsl:template>

  <xsl:template name="calculateFibonacci">
    <xsl:param name="index"/>
    <xsl:param name="low"  select="1"/>
    <xsl:param name="high" select="1"/>
    <xsl:choose>
      <xsl:when test="$index &lt;= 1">
        <xsl:value-of select="$low"/>
      </xsl:when>
      <xsl:otherwise>
        <xsl:call-template name="calculateFibonacci">
          <xsl:with-param name="index" select="$index - 1"/>
          <xsl:with-param name="low"   select="$high"/>
          <xsl:with-param name="high"  select="$high + $low"/>
        </xsl:call-template>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>

</xsl:stylesheet>

Alternately, you can pass the namespace qualified name of the extension element to the element-available() function to figure out whether or not the extension is available. For example,

  <xsl:template match="value">
    <int>
      <xsl:choose>
        <xsl:when test="element-available('fib:fibonacci')">
          <fib:fibonacci>
            <xsl:value-of select="number(.)"/>
          </fib:fibonacci>
        </xsl:when>
        <xsl:otherwise>
          <xsl:call-template name="calculateFibonacci">
            <xsl:with-param name="index" select="number(.)" />
          </xsl:call-template>
        </xsl:otherwise>
      </xsl:choose>
    </int>
  </xsl:template>

From this point on, the exact details of how you code the extension element in Java are quite implementation dependent. You’ll need to consult the documentation for your XSLT processor to learn how to write an extension element and install it. You cannot use preexisting methods and classes as extension elements. You need to custom code the extension element so it fits in with the processor’s own code.

Caution

Writing an extension element is much more complex than writing an extension function. It requires intimate knowledge of and interaction with the XSLT processor. If at all possible, you should probably use an extension function, perhaps one that returns a node-set, instead of an extension element.

Prev | Up | Next
TrAX | Home | Summary
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 20, 2002


Summary
Prev Chapter 17. XSLT Next

Summary

XSLT is a template-driven environment for defining transformations between one XML document format and another. An XSLT stylesheet contains template rules that are matched against nodes in the input document. When a match is detected, the rule’s template is instantiated and the result added to the result tree. XSLT is itself an XML application that contains numerous elements for specifying exactly which nodes are matched when and what output is generated as a result.

XSLT is also a Turing-complete, functional programming language. Templates can be called by name, and can be called recursively. Parameters can be passed to templates. XSLT is not as convenient to program in as traditional languages like Java, but in its problem domain this approach works extremely well.

TrAX is a standard API for XSLT processors that is part of Java 1.4 and later and is bundled with most current XSLT processors for Java including Saxon, Xalan, and jd.xslt. TrAX follows the abstract factory design pattern. A TransformerFactory object creates Transformer objects from stylesheets. The Transformer objects transform XML documents in a variety of representations according to the instructions in their stylesheets. Both XSLT stylesheets and input XML documents are initially wrapped by a Source object. The target for output is wrapped by a Result object. TrAX includes a number of standard source and result implementations for different models including SAX, DOM, and streams.

As well as integrating XSLT into your Java programs, you can integrate Java into your XSLT stylesheets through extension elements and functions. The extension-element-prefixes attribute specifies which namespace prefixes are mapped to extension elements. The namespace URIs are used to find the necessary .class files in the processor’s class path. The element-available() and function-available() functions and the xsl:fallback element test for the presence of extension functions and elements so you can write portable stylesheets that work with or without the extensions. Once you’ve set this up, you can use the extension elements and functions just like standard XSLT elements and functions.

Prev | Up | Next
Extending XSLT with Java | Home | Appendix A. XML APIs Quick Reference
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified June 04, 2002


Colophon
Prev

Colophon

This entire book is being written in XML from start to finish. The specific XML application used is DocBook 4.2.0. I use the jEdit text editor on Windows and Linux to write. XInclude is used to merge the individual chapters and examples together. Michael Kay's SAXON XSLT processor and Norm Walsh’s XSL stylesheets for Docbook 1.52.2 produce the HTML and XSL-FO output. I use FOP 0.20.4 to convert the XSL-FO files to PDFs. The DocBook source files were pulled into Adobe FrameMaker to layout the printed book.

Prev | Up
Index | Home
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified August 05, 2002


Index
Prev Next

Index

Prev | Up | Next
Recommended Reading | Home | Colophon
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 15, 2002


Preface
Prev Next

Preface

Table of Contents

Who You Are
What You Need to Know
What You Need to Have
How to Use This Book
The Online Edition
Some grammatical notes
Contacting the Author

One night five developers, all of whom wore very thick glasses and had recently been hired by Elephants, Inc., the world’s largest purveyor of elephants and elephant supplies, were familiarizing themselves with the company’s order processing system when they stumbled into a directory full of XML documents on the main server. “What’s this?” the team leader asked excitedly. None of them had ever heard of XML before so they decided to split up the files between them and try to figure out just what this strange and wondrous new technology actually was.

The first developer, who specialized in optimizing Oracle databases, printed out a stack of FMPXMLRESULT documents generated by the FileMaker database where all the orders were stored, and began poring over them. “So this is XML! Why, it’s nothing novel. As anyone can see who’s able, an XML document is nothing but a table!”

“What do you mean, a table?” replied the second programmer, well versed in object oriented theory and occupied with a collection of XMI documents that encoded UML diagrams for the system. “Even a Visual Basic programmer could see that XML documents aren’t tables. Duplicates aren’t allowed in a table relation, unless this is truly some strange mutation. Classes and objects is what these document are. Indeed, it should be obvious on the very first pass. An XML document is an object and a DTD is a class.”

“Objects? A strange kind of object, indeed!” said the third developer, a web designer of some renown, who had loaded the XHTML user documentation for the order processing system into Mozilla. “I don’t see any types at all. If you think this is an object, then it’s your software I refuse to install. But with all those stylesheets there, it should be clear to anyone not sedated, that XML is just HTML updated!”

“HTML? You must be joking” said the fourth, a computer science professor on sabbatical from MIT, who was engrossed in an XSLT stylesheet that validated all the other documents against a Schematron schema. “Look at the clean nesting of hierarchical structures, each tag matching its partner as it should. I’ve never seen HTML that looks this good. What we have here is S-expressions, which is certainly nothing new. Babbage invented this back in 1882!”

“An S expression?” queried the technical writer, who was occupied with documentation for the project written in DocBook. “Maybe that means something to those in your learned profession. But to me, this looks just like a FrameMaker MIF file. However, locating the GUI is taking me awhile.”

And so they argued into the night, none of them willing to give an inch, all of them presenting still more examples to prove their points, none of them bothering to look at the others’ examples. Indeed, they’re probably still arguing today. You can even hear their shouts from time to time on xml-dev. Their mistake, of course, was in trying to force XML into the patterns of technologies they were already familiar with rather than taking it on its own terms. XML can store data, but it is not a database. XML can serialize objects, but an XML document is not an object. Web pages can be written in XML, but XML is not HTML. Functional (and other) programming languages can be written in XML, but XML is not a programming language. Books are written in XML, but that doesn’t make XML desktop publishing software.

XML is something truly new that has not been seen before in the world of computing. There have been precursors to it, and there are always fanatics who insist on seeing XML through database (or object, or functional, or S-expression) colored glasses. But XML is none of these things. It is something genuinely unique and new in the world of computing; and it can only be understood when you’re willing to accept it on its own terms, rather than forcing it into yesterday’s pigeon holes.

There are a lot of tools, APIs, and applications in the world that try to pretend XML is something more familiar to programmers; that it’s just a funny kind of database, or just like an object, or just like remote procedure calls. These APIs are occasionally useful in very restricted and predictable environments. However, they are not suitable for processing XML in its most general format. They work well in their limited domains, but they fail when presented with XML that steps outside the artificial boundaries they’ve defined. XML was designed to be extensible, but it’s a sad fact that many of the tools designed for XML aren’t nearly as extensible as XML itself.

This book is going to show you how to handle XML in its full generality. It pulls no punches. It does not pretend that XML is anything except XML, and it shows you how to design your programs so that they handle real XML in all its messiness: valid and invalid, mixed and unmixed, typed and untyped, and both all and none of these at the same time. To that end, this book focuses on those APIs that don’t try to hide the XML. In particular, there are three major Java APIs that correctly model XML, as opposed to modeling a particular class of XML documents or some narrow subset of XML. These are:

  • SAX, the Simple API for XML

  • DOM, the Document Object Model

  • JDOM, a Java native API

These APIs are the core of this book. In addition I cover a number of preliminaries and supplements to the basic APIs including:

  • XML syntax

  • DTDs, schemas, and validity

  • XPath

  • XSLT and the TrAX API

  • JAXP, a combination of SAX, DOM, and TrAX with a few factory classes

And, since we’re going to need a few examples of XML applications to demonstrate the APIs with, I also cover XML-RPC, SOAP, and RSS in some detail. However, the techniques this book teaches are hardly limited to just those three applications.

Who You Are

This book is written for experienced Java programmers who want to integrate XML into their systems. Java is the ideal language for processing XML documents. Its strong Unicode support in particular made it the preferred language for many early implementers. Consequently, more XML tools have been written in Java than in any other language. More open source XML tools are written in Java than in any other language. More programmers process XML in Java than in any other language.

Processing XML with Java will teach you how to:

  • Save XML documents from applications written in Java

  • Read XML documents produced by other programs

  • Search, query, and update XML documents

  • Convert legacy flat data into hierarchical XML

  • Communicate with network servers that send and receive XML data

  • Validate documents against DTDs, schemas, and business rules

  • Combine functional XSLT transforms with traditional imperative Java code

This book is meant for Java programmers who need to do anything with XML. It teaches the fundamentals and advanced topics, leaving nothing out. It is a comprehensive course in processing XML with Java that takes developers from little knowledge of XML to designing sophisticated XML applications and parsing complicated documents. The examples cover a wide range of possible uses including file formats, data exchange, document transformation, database integration, and more.

What You Need to Know

This is not an introductory book with respect to either Java or XML. I assume you have substantial prior experience with Java and preferably some experience with XML. On the Java side, I will freely use advanced features of the language and its class library without explanation or apology. Among other things, I assume you are thoroughly familiar with:

  • Object oriented programming including inheritance and polymorphism

  • Packages and the CLASSPATH. You should not be surprised by classes that do not have main() methods or that are not in the default package.

  • I/O including streams, readers, and writers. You should understand that System.out is a horrible example of what really goes on in Java programs.

  • The Java Collections API including hash tables, maps, sets, iterators, and lists.

In addition, in one or two places in this book I’m going to use some SQL and JDBC. However, these sections are relatively independent of the rest of the book; and chances are if you aren’t already familiar with SQL, then you don’t need the material in these sections anyway.

What You Need to Have

XML is deliberately architecture, platform, operating system, GUI, and language agnostic (in fact, more so than Java). It works equally well on Mac OS, Windows, Linux, OS/2, various flavors of Unix, and more. It can be processed with Python, C++, Haskell, ECMAScript, C#, Perl, Visual Basic, Ruby, and of course Java. No byte order issues need concern you if you switch between PowerPC, X86, or other architectures. Almost everything in this book should work equally well on any platform that’s capable of running Java.

Most of the material in this book is relatively independent of the specific Java version. Java 1.4 bundles SAX, DOM, and a few other useful classes into the core JDK. However, these are easily installed in earlier JVMs as open source libraries from the Apache XML Project and other vendors. For the most part, I used Java 1.3 and 1.4 when testing the examples; and it’s possible that a few classes and methods have been used that are not available in earlier versions. In most cases, it should be fairly obvious how to backport them. All of the basic XML APIs except TrAX should work in Java 1.1 and later. TrAX requires Java 1.2 or later.

Prev | Up | Next
Processing XML with Java | Home | How to Use This Book
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified July 12, 2002


How to Use This Book
Prev Preface Next

How to Use This Book

This book is organized as an advanced tutorial that can also serve as a solid and comprehensive reference. The first chapter covers the bare minimum material needed to start working with XML, though for the most part this is intended more as a review for readers who’ve already read other, more basic books than as a comprehensive introduction. The second chapter introduces RSS, XML-RPC, and SOAP, the XML applications we’ll be using for examples in the rest of the book. This is followed by two chapters on generating XML from your own programs (a subject which is all too often presented as a lot more complicated than it actually is). The first covers generating XML directly from code. The second covers converting legacy data in other formats to XML. The remaining bulk of the book is devoted to the major APIs for processing XML:

  • The event based SAX API

  • The tree-based DOM API

  • The tree-based JDOM API

  • XPath APIs for searching XML documents

  • The TrAX API for XSLT processing

Finally, the book finishes with an appendix providing quick references to the main APIs.

If you have limited experience with XML, I suggest you read at least the first five chapters in order. From that point forward, if you have a particular API preference, you may begin with the part covering the major API you’re interested in:

  • Chapters 6-8 cover SAX

  • Chapters 9-13 cover DOM

  • Chapters 14 and 15 cover JDOM

Once you’re comfortable with one or more of these APIs, you can read Chapters 16 and 17 on XPath and XSLT. However, those APIs and chapters do require some knowledge of at least one of the three major APIs.

Prev | Up | Next
Preface | Home | The Online Edition
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 21, 2002


The Online Edition
Prev Preface Next

The Online Edition

The entire book is available online in plain vanilla HTML at my Cafe con Leche web site. You can find it at http://www.cafeconleche.org/books/xmljava/. Every word of this book is there. Nothing has been held back or left out. I do hope you’ll also find the printed book useful and choose to buy it—It’s certainly cheaper than the paper and toner you’d use up printing out all 1000+ pages from your laser printer—but you are by no means obligated to do so. My goal is to make this material as broadly available and useful as possible.

The online version has no protection other than copyright law and your own good will. You don’t need to register to read it, or download some special electronic key that becomes invalid when you buy a new laptop, and that probably won’t run on Linux or a Mac in the first place. I want people to read and use this book. I do not want to put up silly roadblocks that make it less useful than it could be. I will ask that as a courtesy you do not republish the online edition on your own server. Doing so makes it extremely difficult for me to keep the book up-to-date. If you want to save a few pages on your laptop so you can read this book on an airplane, I don’t really mind. However, please don’t pass out your own copies to anyone else. Instead refer your friends and colleagues to the web site or the printed book.

Prev | Up | Next
How to Use This Book | Home | Some grammatical notes
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 11, 2002


Some grammatical notes
Prev Preface Next

Some grammatical notes

The rules of English grammar were laid down, written in stone, and encoded in the DNA of grammar school teachers long before computers were invented. Unfortunately, this means that sometimes I have to decide between syntactically correct code and syntactically correct English. When forced to do so, English normally loses. This means that sometimes a punctuation mark appears outside a quotation mark when you’d normally expect it to appear inside, a sentence begins with a lower case letter, or something similarly unsettling occurs. For the most part, I’ve tried to use various typefaces to make the offending phrase less jarring. In particular,

  • Italicized text is used for emphasis, the titles of books and other cited works, words in languages other than English, words used to refer to the words themselves (for example, Booboisie is a very funny word.), the first occurrence of an important term, Java system properties, host names, resolvable URLs.

  • Monospaced text is used for XML and Java source code, namespace URLs, system prompts, and program output

  • Italicized monospaced text is used for pieces of XML and Java source code that should be replaced by some other text

  • Bold text is used for emphasis

  • Bold monospaced text is normally used for literal text the user types at a command line, as well as for emphasis in code.

It’s not just English grammar that gets a little squeezed either. The necessities of fitting code onto a printed page rather than a computer screen have occasionally caused me to deviate from the ideal Java coding conventions. The worst problem is line length. I can only fit 65 characters across the page in a line of code. To try and make maximum use of this space, I indent each block by two spaces and indent line continuations by one space, rather than the customary four spaces and two spaces respectively. Even so, I still have to break lines where I’d otherwise prefer not to. For example, I originally wrote this line of code for Chapter 4:

result.append("          <Amount>" + amount + "</Amount>\r\n");

However, to fit it on the page I had to split it into two pieces like this:

result.append("          <Amount>");
result.append(amount + "</Amount>\r\n");

This case isn’t too bad, but sometimes even this isn’t enough and I have to remove indents from the front of the line that would otherwise be present. For example, this occasionally forces the indentation not to line up as prettily as it otherwise might, as in this example from Chapter 3

      wout.write(
    "xmlns='http://namespaces.cafeconleche.org/xmljava/ch3/'\r\n"
      );

The silver lining to this cloud is that sometimes the extra attention code gets when I’m trying to cut down its size results in better code. For example, in Chapter 4 I found I needed to remove a few characters from this line:

OutputStreamWriter wout = new OutputStreamWriter(out, "UTF8");

On reflection I realized that nowhere did the program actually need to know that wout was an OutputStreamWriter as opposed to merely a Writer. Thus I could easily rewrite the offending line as follows:

Writer wout = new OutputStreamWriter(out, "UTF8");

This follows the general object oriented principle of using the least specific type that will suit. This polymorphism makes the code more flexible in the future should I find a need to swap in a different kind of Writer.

Prev | Up | Next
The Online Edition | Home | Contacting the Author
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified September 02, 2002


Contacting the Author
Prev Preface Next

Contacting the Author

I always enjoy hearing from readers whether with general comments, specific ways I could improve the book, or questions related to the book’s subject matter. Since this book is being published in its entirety online, it is possible for me to reprint at least the online edition much faster than can be done with a traditional paper book. Thus corrections and errata are especially helpful since there’s a real chance for me to fix them. Before sending in a correction, please do check the online edition to see if I already know about and have fixed the problem.

Please send all comments, inquiries, bouquets, and brickbats to elharo@metalab.unc.edu. I get a lot of e-mail, so I can’t promise to answer them all; but I do try. It’s helpful if you use a subject line that clearly identifies yourself as a reader of this book. Otherwise, your message may get accidentally misidentified as spam I don't want or bulk mail I don’t have time to read and get dropped in the bit bucket before I see it. Also, please make absolutely sure that your message uses the correct Reply-to address and that the address will be valid for at least several months after you send the message. There’s nothing quite so annoying as taking an hour or more to compose a detailed response to an interesting question, only to have it bounce because the reader sent the e-mail from a public terminal or changed their ISP. But please do write to me. I want to hear from you.

--
Elliotte Rusty Harold
Brooklyn, New York
June 7, 2002

Prev | Up | Next
Some grammatical notes | Home | Acknowledgements
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified May 21, 2002


Acknowledgements
Prev Next

Acknowledgements

Thomas Marlin provided me with the original Latin text of the Fibonacci problem you’ll find in Chapter 3.

Jason Hunter’s encyclopedic knowledge of the Java Servlet API was essential to the design and execution of the servlet code in this book. Donald Sizemore helped me get my servlets installed and running on IBiblio.

Luke Tymowski provided some of the RSS examples and helped me debug various problems with my Cobalt Qube.

Bruce Eckel and Chuck Allison helped me decipher the relative capabilities of Java and C++. Bruce Eckel also helped out with Python. Matt Sergeant and Brendan McKenna helped out with Perl. Philip Nelson, Robert A. Casola, and Rob Smith helped with Visual Basic. None of these people necessarily agree with what I wrote about those relative capabilities; (In fact, more often than not they vehemently disagree; de linguis non disputandum est); but I couldn’t have done it without them.

Although this is the sixth book I’ve written about XML, it is the first one I’ve written in XML. That could not have happened without Norm Walsh’s Docbook DTD and XSL stylesheets for Docbook.

Many people helped out with comments, corrections, and suggestions. These include Payman Aliverdi, Sergey Astakhov, Janek Bogucki, Dagmar Buggle, William Chang, Richard Dedeyan, Paul Duffin, Lacey Anne Edwards, Peter Elliott, Paul Erion, Bernard Farrell, Wei Gao, Scott Harper, Stefan Hässig, Martin Henke, Markus Jais, Oliver Korpilla, Igor Kostjuhin, Alexander Krumpholz, Wes Kubo, Ramnivas Laddad, Manos Laliotis, Ian Lea, Frank Lee, Ray Leyva, Rob Lugt, Richard Monson-Haefel, Gary Nichols, James Orenchak, Milton Quiroga, Aron Roberts, Carlo Rossi, Raheem Rufai, Arthur E. Salwin, Peter Sellars, Diana Shannon, Andrew Shebanow, and Fred Trimble. Mike Blackstone deserves special thanks for his copious notes.

Mike Champion, Andy Clark, Robert W. Husted, Anne T. Manes, Ron Weber, and John Wegis did yeomanlike service as technical reviewers. Their comments substantially improved the book.

As always, the folks at the Studio B literary agency were extremely helpful at all steps of the process. David Rogelberg, Sharon Rogelberg, and Stacey Barone should be called out for particular commendation.

This is my first book for Addison-Wesley, but it’s not going to be my last. They were all wonderful people to work with, and I look forward to working with them again. Mary T. O’Brien shepherded this book from contract to completion. Alicia Carey ably managed submissions and communications.

Finally, as always, my biggest thanks are due to my wife Beth without whose love and understanding this book could never have been completed.

Prev | Up | Next
Contacting the Author | Home | Chapter 1. XML for Data
Copyright 2001, 2002 Elliotte Rusty Harold | elharo@metalab.unc.edu | Last Modified October 15, 2002