result
This article refers to dap version 0.1.6

Lesson 2. Accessing Data

In this tutorial, we'll build a simple 'data access page', demonstrating the dap way of data interpretation.

Dap primarily accesses data over HTTP, using XMLHttpRequest Javascript object (so-called AJAX technology). Dap core provides a set of converters (#-converters) which "convert" an URL string into data, received from that URL and interpreted in a specific way. All these core #-converters work with XML data files, so the datasource must be able to provide data as XML. Custom #-converters may be implemented in extension libraries to deal with other file formats or transports.

I'd recommend that you monitor HTTP traffic of this tutorial (using FireBug in Firefox, or a similar tool), to see the HTTP requests genegated by the page, and XML responses from the datasource.

First, we need a datasource. Meet Bazamagaza.

In this tutorial, we will use a datasource, similar to that used by mariachi.ru. It is an MS SQL Server database coupled with a simple asp.net gate. The gate's only function is to translate HTTP requests into SQL stored proc calls and return the result set as XML, thus providing communication between SQL Server and XML/HTTP client. The whole business logic is incorporated into SQL database.

The data in this datasource is a subset of the snapshot of actual mariachi's data on one of the days. It is partially in Russian. If you don't know Russian, you will hardly learn anything new about saxophones; but for learning dap, it will hopefully not be a problem.

The gate accepts queries as parameterized HTTP (GET or POST) requests, and returns ordinary XML datasets. The request must provide at least one required param: _(underscore) — it specifies the name of the stored procedure to be executed. Depending on the specified stored procedure, other params may be required.

This datasource's name is Bazamagaza.

Bazamagaza's internal SQL logic is beyond this lesson's subject — we'll use it as an XML/HTTP blackbox. Here are some examples of HTTP requests it accepts:

All these responses have basically the same format: a top-level NewDataSet element, having a set of Table elements, each of those Table elements being actually a data row (by some reason, .NET returns dataset rows as Table elements). This is a typical XML rowset as it comes from MS SQL Server by DataSet.GetXml() method.

Second, we need a markup template

In this lesson we'll build a simple assortment navigation interface. The interface will allow customers to select the type(class) of an instrument, its vendor(brand), and also to select the cheapest, or the most ranked, or the most expensive of them.

Since the discussion of HTML design trends is far beyond the subject of this tutorial, let's confine ourself with these fairly simple templates:

HTML:
	
<h3>Classes</h3>
<div class="nav">
 <ul>
  <li><span>Flutes</span></li>
  <li><span>Saxophones</span>
   <ul>
    <li class="over"><span>Alto-Saxophone</span></li>
    <li><span>Baritone-Saxophone</span></li>
    <li class="chosen"><span>Tenor-Saxophone</span></li>
    <li><span>Soprano-Saxophone</span></li>
   </ul>
  </li>
  <li><span>Trumpets</span></li>
  <li><span>Trombones</span></li>
 </ul>
</div>

<h3>Brands</h3>
<div class="nav">
 <ul>
  <li>Lorem</li>
  <li>Ipsum</li>
  <li class="chosen">Chosen</li>
  <li>Consectetuer</li>
  <li class="over">Hovered</li>
 </ul>
</div>

<h3>Items</h3>
<div class="items">
 <div>
  <h3>Yamaha YTS-62 Professional</h3>
  <i>tenor saxophone</i>
  <p>A short description of the instrument.</p>
  <p>Price: <b>123 000</b> p.</p>
 </div>
 ...
 </div>
</td>



CSS styles

td.menu{width:160px}
h3{font:bold 16px sans-serif}

.nav ul{padding:2px 0 2px 8px; color:#666}
.nav li{padding:4px; margin:0; list-style:none; white-space:nowrap; font:normal 12px sans-serif}
.nav li.over,.nav li>span.over{color:#000}
.nav li.chosen,.nav li>span.chosen{color:#f80}

.nav li>span.open{text-decoration:underline}

.items{margin:8px; border:1px solid #ccc;height:500px; width:520px; overflow:auto}
.items>div{border:1px solid #ccc; padding:8px; margin:8px}
.items  h3{font:bold 16px sans-serif; color:#f80}
.items p{font:normal 12px sans-serif}

Items

Yamaha YTS-62 Professional

tenor saxophone

A short description of the instrument.

Price: 123 000 p.

Trevor J. James Horn Classic

alto saxophone

A short description of the instrument. By the way, very good saxophones — Trevor J. James, and quite unexpensive.

Price: 56 000 p.

P. Mauriat PMXA-67R

alto saxophone

Another great sax. Really. 'R' means 'Rolled tone holes'.

Price: 234 000 p.

This is a static imitation of what we expect the application to look like.

Now dap it !

Once the XML datasource and the layout template are ready, we can bind them together with dap.

To get data from a given URL, we use the dap core #dat converter. It is specifically designed to deal with such datasets.

The * mapper generates output markup for each given row, using the node content as template.

<h3>Classes</h3>
<div class="nav">
 <ul d="*:#dat=url =/samples/bazamagaza/? @_=AllClasses">
  <li d="! classname"></li>
 </ul>
</div>

<h3>Brands</h3>
<div class="nav">
 <ul d="*:#dat=url =/samples/bazamagaza/? @_=AllBrands">
  <li d="! brand"></li>
 </ul>
</div>

The whole lists are definitely too long. It is a good idea to limit the list size by some criterions:

  1. Not to show all the instrument classes at once, but organize them into expandable tree.
  2. Only show the brands having instruments of currently selected class in stock.

Expandable tree

Unlimited-level trees can be easily defined in dap using template recursion. The idea behind template recursion is very simple: the template reference is stored in a data or status entry, which is invoked inside the template. The node's own template content (as an array of its child nodes) is addressed in a dap rule by an empty token. In the example below, $subclasses status entry is used to reference subtree template.

Expand/collapse states of each subtree can be controlled by a status enry, $open in the example below. Each subtree is executed or not, depending on value of that entry. On user events, $open can be toggled by - mapper, and tested by ? mapper. Each subtree will have its own status scope and own $open entry within it, initially empty in the example. On user event on a given subtree, its $open entry is toggled, allowing the subtree to execute. Next event over the opened subtree toggles its $open back to false (equivalent to empty), and thus shuts the subtree. The top-level $open status entry is initialized by non-empty value (say, '1') to allow the initial execution of the tree's root.

Bazamagaza allows to select subclasses of a given class by ClassesOf @class request, for example: /samples/bazamagaza/?_=ClassesOf&class=WS returns the subclasses of class 'WS' (Winds → Saxophones), namely: 'WSA' (alto saxophone), 'WSB' (baritone saxophone), etc.

<h3>Classes</h3>
<div class="nav" d="$subclasses ;  $open=1">
 <span d="! classname; ui" u="-$open"></span>
 <ul d="? $open@; *:#dat=url =/samples/bazamagaza/? @_=ClassesOf class">
  <li d=" $open=; ! $subclasses"></li>
 </ul>
</div>

Classes

Dependencies

To limit the brands list size, let's only pick the brands having instruments of the specified class in stock. Bazamagaza provides the BrandsOf @class request for that. For example, /samples/bazamagaza/?_=BrandsOf&class=WSA returns only those brands having alto saxophones in stock.

To pass selected class from class selection list to brands list, a $class status entry is introduced in a scope observable for both lists — that is, in the scope of their common ancestor. Every time a user selects another class, $class entry changes and forces the brands list rebuild.

Given the two selection criterions — the class and the brand, we are ready to pick the best matching items in stock — using the BestOf @class @brand @critn @count request. The max count of returned rows and "the best"-ness criterion can also be specified. Let's limit the count to a dozen of items, and assume "the best" means "the best ranked". For example, /samples/bazamagaza/?_=BestOf&class=WST&brand=P.+Mauriat&critn=ranked&count=12 returns the top dozen best ranked P. Mauriat tenor saxophones.

Another good idea is to store the datasource base URL in a data or status entry, too. This will shorten rules and eliminate the need to manually update multiple references to the datasource in the case the URL is changed.

<table d=" $src=/samples/bazamagaza/?">
 <tbody>
  <tr d=" $class= $brand= $classname=instrument">
   <td class="menu">
    <h3>Classes</h3>
    <div class="nav" d="$subclasses ;  $open=1">
     <span d="! classname; ui" u="-$open;  class$ classname$ $brand="></span>
     <ul d="? $open@; *:#dat=url $src@ @_=ClassesOf class">
      <li d=" $open=; ! $subclasses"></li>
     </ul>
    </div>
   </td>
   <td class="menu">
    <h3>Brands</h3>
    <div class="nav">
     <ul d="? $class@; *:#dat=url $src@ @_=BrandsOf $class">
      <li d="! brand; ui" u=" brand$"></li>
     </ul>
    </div>
   </td>
   <td>
    <h3 d="!=space =The =best $brand $classname">s are:</h3>
    <ul d="*:#dat=url $src@ @_=BestOf $class $brand @critn=ranked @count=12">
     <li>
       <span d="!=space brand display series classname"></span> @<b d="! price"> RUR</b>
    </ul>
   </td>
  </tr>
 </tbody>
</table>

s in stock are:

  • @ RUR

To distinguish the chosen items in the list, let's highlight them using a dedicated CSS class. Using CSS, styles, colors etc is somewhat HTML-specific stuff, so this sort of functionality is implemented in a separate dap extension library — htm.xml, a part of dap Starter Kit.

We use htm.? mapper from Starter-Kit/htm.xml. This mapper marks the node with CSS classes specified by the token aliases, if the token value is true/present, otherwise removes those CSS classes from the node. To test if the list item is 'chosen' or not, we compare its datarow key field value against the 'currently chosen' value of the corresponding status variable. The comparison for equality is performed by dap core eq flattener.

The BestOf request allows to select items, assuming that "the best" might mean "the most rated", "the cheapest", or "the most expensive". This criterion is specified by @critn request param. Let's implement the criterion selector using rudimentary HTML select element.

Let's also wrap the items with the planned markup. To format prices, we use fo.num convertor from starter-kit/formats.xml dap extension library.

<table d=" $src=/samples/bazamagaza/?; 
:#lib @htm=/0.1.6/starter-kit/htm.xml @mf=/0.1.6/starter-kit/formats.xml">
 <tbody>
  <tr d=" $critn= $class= $brand= $classname=instrument">
   <td class="menu">
    <h3>Classes</h3>
    <div class="nav" d="$subclasses ;  $open=1">
     <span d="! classname; htm.?@chosen=eq class $class; ui" u="-$open;  class$ classname$ $brand="></span>
     <ul d="? $open@; *:#dat=url $src@ @_=ClassesOf class">
      <li d=" $open=; ! $subclasses"></li>
     </ul>
    </div>
   </td>
   <td class="menu">
    <h3>Brands</h3>
    <div class="nav">
     <ul d="? $class@; *:#dat=url $src@ @_=BrandsOf $class">
      <li d="! brand; htm.?@chosen=eq brand $brand; ui" u=" brand$"></li>
     </ul>
    </div>
   </td>
   <td>
    <h3>The 
     <select d="ui@change; * :nvp,usc@critn,title=ranked:most%20rated;cheap:cheapest;pathos:most%20expensive"
        u="htm.%$critn">
      <option d="! title; attr critn@value"></option>
     </select>
     &nbsp;
    <span d="!=space $brand $classname">s in stock are:</span>
    </h3>
    <div d="*:#dat=url $src@ @_=BestOf $class $brand $critn @count=12" class="items">
     <div>
      <h3 d="!=space brand display series"></h3>
      <i d="! classname"></i>
      <p d="!=? annotation :usc=Annotation%20not%20provided"></p>
      <p>Price: <b d="! price:fo.num"></b> RUR</p>
     </div>
    </div>
   </td>
  </tr>
 </tbody>
</table>
	

The   s in stock are:

Price: RUR

Non-destructive element masking

Fair rebuilds of DOM in response to user activity are not always desired.

If you were monitoring HTML traffic while studying the example above, you might notice, that every time you expand a subtree in the classes menu, a new HTTP request is executed to get data, even if that data has already been acquired before. For example, if you expand saxophones for the first time, a request is executed to get saxophones subclasses. XML response gives them to us: alto saxophone, baritone saxophone, soprano and tenor. Okay. Next click on saxophones collapses saxophones subclasses. If you click saxophones the third time, it expands again — and again executes the same query as on the first click. Each time a status variable is changed, dependent pieces of DOM are rebuilt new, and their d-rules are executed again and again. In terms of performance it is not a big deal — thanks to HTTP caching, subsequent identical requests will not generate actual network activity, and will fetch from cache (provided that cache policies are appropriately configured).

However, if a dependent node updates, the state of its old childnodes is lost. That is, if you expand winds, and then expand say, flutes and trumpets, and after that collapse and re-expand winds — flutes and trumpets will not be expanded in those "reborn" winds — they will be collapsed as if you never expanded them. Sometimes, such a behavior is not what you might want.

If a node's state is valuable, even when the node is not visible — it should be preserved. Instead of actually removing the node from the DOM, we need to visually hide it. Yes, good ole display:none, visibility:hidden, etc. Since this way of node handling is not related with DOM manipulation, and is sort of HTML/CSS specific stuff, it is implemented in an external library — starter-kit/fx.xml.

<table d=" $src=/samples/bazamagaza/?;
; :#lib @htm=/0.1.6/starter-kit/htm.xml @mf=/0.1.6/starter-kit/formats.xml @fx=/0.1.6/starter-kit/fx.xml"
 >
 <tbody d="@dropdown:fx.behavior= @toggle=alpha;damp:100;top @show=alpha:0;top:0 @hide=alpha:100;top:100">
  <tr d=" $critn= $class= $brand= $classname=instrument">
   <td class="nav">
    <h3>Classes</h3>
    <div d="$subclasses ;  $open=1; :fx.event @toggle" class="classes">
     <span d="? $open:!@; ! classname; ui" u=" class$ classname$ $brand=;  $open=1; "></span>
     <span d="? $open@; ! classname; ui" u=" class$ classname$ $brand=; fx.! toggle"></span>
     <div d="? $open@" style="overflow:hidden"> 
      <ul d="fx.? dropdown@ toggle; *:#dat=url $src@ @_=ClassesOf class">
       <li d=" $open=; :fx.event @toggle; ! $subclasses"></li>
      </ul>
     </div>
    </div>
   </td>
   <td class="menu">
    <h3>Brands</h3>
    <div class="nav">
     <ul d="? $class@; *:#dat=url $src@ @_=BrandsOf $class">
      <li d="! brand; htm.?@chosen=eq brand $brand; ui" u=" brand$"></li>
     </ul>
    </div>
   </td>
   <td d=":fx.event @expand-all @collapse-all">
    <h3>The 
     <select d="ui@change; * :nvp,usc@critn,title=ranked:most%20rated;cheap:cheapest;pathos:most%20expensive" u="htm.%$critn">
      <option d="! title; attr critn@value"></option>
     </select>
     &nbsp;
     <span d="!=space $brand $classname">s in stock are:</span>
    </h3>
    <div d="*:#dat=url $src@ @_=BestOf $class $brand $critn @count=12" class="items">
     <div d=":fx.event @toggle">
      <h3 d="!=space brand display series; ui" u="fx.! toggle"></h3>
      <div style="overflow:hidden">
       <div d="fx.? dropdown@ toggle expand-all@show collapse-all@hide">
        <i d="! classname"></i>
        <p d="!=? annotation :usc=Annotation%20not%20provided"></p>
        <p>Price: <b d="! price:fo.num"></b> RUB</p>
       </div>
      </div>
     </div>
    </div>
    <span d="ui" u="fx.! expand-all">expand all</span> | <span d="ui" u="fx.! collapse-all">collapse all</span>
   </td>
  </tr>
 </tbody>
</table>
	

The   s in stock are:

Price: RUB

expand all | collapse all

The actual e-shop at http://dapmx.org/apps/bazamagaza is a bit more complex (yet is pure dap) and will be studied later.


DAPMX 0.1