Blog moved

All change.  I have started posting in my new blog

http://guyboertje.github.io/

Thanks

Flexifields Rails Plugin Published on Github

I have recently published a Ruby on Rails plugin on Github. The requirement came about because I have been doing a lot of work with the Adva CMS rails application (look into the adva system at github). Adva uses engines to modularise the various CMS functional areas. Being a content management system, adva has a data model for Content and a sub-class of this for an Article. Articles or Content have three attributes that can store text each with an implicit label: title, body and excerpt. I needed more explicitly labelled attributes to store extra text for a 'flavor' of Article - take a Recipe as an example of a flavor of Article and let us say that a CMS admin wants to define extra attributes for prep_time, cooking_time and serves_portions so that a CMS editor can supply these extra texts outside of the main body of the recipe in a consistent way. Note that the names of these extra attributes are not known at design time so the data model can't be extended to include these attributes. I decided to write a plugin (well its turned out to be a mini-engine for rails 2.3 because I decided not to require the plugin user to use generators to copy the models and routes into the main rails app) that adds the ability to have extra attributes as a has_one association off of any model. Each flavor is modelled as a Flexifield Definition and each label for a given flavor is modelled as a Flexifield Definition Entry. Using this plugin I can have different flavors of Article that can store up to 16 differently labelled texts per Article record.

http://github.com/guyboertje/has_flexiblefields/tree/master

The next steps will be to look into using it in Adva. In adva the administrative routes/pages all point to files below an admin folder. So adding support for CRUD on the labels (called FlexifieldDefEntries) via the standard Adva admin forms will require a few folder and route changes.

Also, on the adva group and IRC, we debated whether it made more sense to create a separate engine for each flavor of Article - I argued that I was not modelling a Recipe or Car Review entity but an Article entity that is specialised, hence the extra attributes and not a new engine. It could also be argued that these flavors of Article require that certain common (to each flavor) bits of text are removed from needing to be (randomly placed) in the body and are stored separately.

An Adobe Flex Demo

Below is a link to a flex application that I created. Please note, it is a demo, not production ready code. I would welcome comments.

Warranty Submission Demo

You need to drag (or dbl click) something from the upper middle tree before you can drag from the upper right tree. For the final design the contents of the right most tree will change dependent on the product being claimed for.

There are a few very interesting things going on in it.

• Drag and drop - drag from any entry in the top half and drop into the bottom half
  
• Central property editor grid that is a single point of edit, normally a column in a grid has the same item editor down all rows but this component has different editors for each row.
  
• Local storage of edits - your changes are remembered.
• Different tab navigator - actually a skinned radio button bar acting on a single box

There is some basic skinning and embedding of fonts. Although this was a Flex 2.0.1 design originally, I have recompiled it for Flex 3.

Ajax (sort of) File Upload using LiveQuery/jQuery

I was building an Ajax only app, the kind that loads a minimal start page and stuffs empty divs with GET responses that are generated by partials just after the page loads. I needed to upload files.

There are many pages on the net explaining how to achieve this using a hidden iframe. To sum up the page uses a form not an XmlHttpRequest to POST the form and the results are loaded in the iframe which is the target for the POST response.

Most folks will not be that interested in the response. I was because the form created or edited a record and the file upload was a picture - part of the record. I needed to get the response, a 'show' partial of the record with a thumbnail out of the iframe and into a visible div.

There are not that many solutions out there (there is a jQuery forms one). I was using LiveQuery to maintain bound events on dynamic content anyway so I decided to use it again. This is what I came up with.

<script>
  $(document).ready(function(){
...
    $('iframe').livequery('load', function() {
      var ifbodyhtml = $(this).contents().find('body').html()
      if(ifbodyhtml.length > 0) {
        $('#showcontact').html( ifbodyhtml )
      }
      return false
    })
  });
</script>
...
      <div id="showcontact">

      </div>
      <iframe id="uploadtarget" name='uploadtarget' src='about:blank'></iframe>

When the page loads the jQuery $(document).ready() function loads various divs with content, one of which is the form in the 'showcontact' div.

The form has a target attribute of 'uploadtarget', so the LiveQuery is set to trap the load event of the iframe. This runs when the page is loaded and the iframe has no content so I've filtered that out. The response is taken from the iframe and put into the div.

That's it.

My Rails Setup

Here I detail my Rails setup on MS Windows 2003.

I have four rails applications, each one served by a Mongrel instance. I have a flatfile intranet served by IIS with Active Directory auth on port 80. I have a reverse proxy IIS plugin which maps requests to server/app through to the appropriate Mongrel. I have a rails plugin which rewrites URLs in server responses to include the appname between the servername and the rest of the URL. All apps persist data to an Oracle XE database.

I use Oracle Maestro to administer the Oracle database. I think it is a great product. I have recently switched to Netbeans 6 to edit the Rails applications. I am very happy with Netbeans.

I also have a Centos 4.4 server for testing. I am testing JRuby and Glassfish, again with an Oracle XE database. While reading about Glassfish I came across Jmaki. Very interesting. More of jmaki in the next issue.

Bringing Jasper Reports, Rails and Rjb together

Whew, talk about difficult. On Windows Server 2003 anyway (business choice, not mine)

Firstly, Rjb (Ruby Java Bridge) requires, absolutely must have, the SDK not just the JRE.

Secondly, get the stuff working in Ruby before trying Rails.

This is what I did.

Install the Rjb Gem. Set JAVA_HOME. Watch out for space in "Program Files" in path, use PROGRAM~1 instead. I installed the SDK to C:\Java to avoid this.

Follow most of HowtoIntegrateJasperReports

I could not get the pipe IO stuff running on Windows - just locked up the server.
I then read anything found on Jasper and found references to PHP Java Bridge with Jasper here, this got me thinking could I use a Ruby Java Bridge to do something like the XmlJasperInterface.java source in Ruby and Rjb without the command line? Eventually. Yes!!

Here is the local ruby...

require 'rjb'

separator = Config::CONFIG['target_os'] =~ /win/ ? ';' : ':'
classpath = [
"./itext-1.3.1.jar",
"./commons-beanutils-1.7.jar",
"./commons-collections-2.1.jar",
"./commons-logging-1.0.2.jar",
"./jasperreports-2.0.4.jar",
"./jasperreports-extensions-1.3.1.jar",
"./jcommon-1.0.0.jar",
"./jdt-compiler-3.1.1.jar",
"./jfreechart-1.0.0.jar",
"./log4j-1.2.9.jar",
"./poi-3.0.1-FINAL-20070705.jar",
"./xalan.jar"
].join(separator)

# xmls is a string representing the @collection.to_xml when on rails

xmls = %Q¬<?xml version="1.0" encoding="UTF-8"?>
<customs-shipping-details>
<customs-shipping-detail>
... lots of attribute nodes
</customs-shipping-detail>
... more customs shipping detail records
</customs-shipping-details>¬

Rjb::load(classpath, ['-Xmx128m'])

j_jre = Rjb::import('net.sf.jasperreports.engine.JRException')
j_jem = Rjb::import('net.sf.jasperreports.engine.JasperExportManager')
j_jfm = Rjb::import('net.sf.jasperreports.engine.JasperFillManager')
j_jp = Rjb::import('net.sf.jasperreports.engine.JasperPrint')
j_jxds = Rjb::import('net.sf.jasperreports.engine.data.JRXmlDataSource')

j_sxis = Rjb::import('org.xml.sax.InputSource')
j_jrxu = Rjb::import('net.sf.jasperreports.engine.util.JRXmlUtils')
j_iosr = Rjb::import('java.io.StringReader')
j_map = Rjb::import('java.util.HashMap')
out = Rjb::import('java.lang.System').out

compiled_design_file_path = './templates/csd_1.jasper'
xpath_filter = '/customs-shipping-details/customs-shipping-detail'
pdf_path = './out102.pdf'
rmap = j_map.new()

xis = j_sxis.new()
xis.setCharacterStream(j_iosr.new(xmls))
jp = j_jfm.fillReport(compiled_design_file_path, rmap,
   j_jxds.new_with_sig('Lorg.w3c.dom.Document;Ljava.lang.String;',
   j_jrxu.parse(xis), xpath_filter))
j_jem.exportReportToPdfFile(jp,pdf_path)

The most frustrating bit was to get the JRXmlDataSource to accept String data and not stream data. The new_with_sig method of Rjb is not perfect; I could not invoke the ByteArrayInputStream constructor with a byte array using the getBytes method of a Java String and the '[B;' signature.

After hunting through the Jasper API I found that I could produce a org.w3c.dom.Document from the static parse method of the JRXmlUtils class. But the parse method needed a org.xml.sax.InputSource object.

Again, the Reader based constructor for the InputSource could not be invoked so I had to use the default and set the Reader source with the setCharacterStream method of the InputSource object. It was easy to use the StringReader object to wrap the XML data string before passing it to the setCharacterStream method.

Moving on to rails.
I created a folder under apps called reports with lib and templates subfolders. I copied the jasper templates into templates and the jar files into lib.

You need this in config/environment.rb ENV['JAVA_HOME'] = "C:\\Java\\jdk1.6.0_05", pointing to your java SDK. Obviously the Rjb gem on the Rails server too.

I created this module in the rails lib folder.

module Jasport
require 'rjb'

separator = Config::CONFIG['target_os'] =~ /win/ ? ';' : ':'
classpath = [
"#{RAILS_ROOT}/app/reports/lib/itext-1.3.1.jar",
"#{RAILS_ROOT}/app/reports/lib/commons-beanutils-1.7.jar",
"#{RAILS_ROOT}/app/reports/lib/commons-collections-2.1.jar",
"#{RAILS_ROOT}/app/reports/lib/commons-logging-1.0.2.jar",
"#{RAILS_ROOT}/app/reports/lib/jasperreports-2.0.4.jar",
"#{RAILS_ROOT}/app/reports/lib/jdt-compiler-3.1.1.jar",
"#{RAILS_ROOT}/app/reports/lib/jfreechart-1.0.0.jar",
"#{RAILS_ROOT}/app/reports/lib/log4j-1.2.9.jar",
#"#{RAILS_ROOT}/app/reports/lib/jasperreports-extensions-1.3.1.jar",
#"#{RAILS_ROOT}/app/reports/lib/jcommon-1.0.0.jar",
#"#{RAILS_ROOT}/app/reports/lib/poi-3.0.1-FINAL-20070705.jar",
"#{RAILS_ROOT}/app/reports/lib/xalan.jar"
].join(separator)

Rjb::load(classpath, ['-Xmx128m'])

class ReportGenerator

def return_pdf(compiled_design_name, xpath_filter, xml_data)
  compiled_design_file_path = "#{RAILS_ROOT}/app/reports/templates/" + compiled_design_name + '.jasper'
  @xis.setCharacterStream(@j_iosr.new(xml_data))
  @jp = @j_jfm.fillReport(compiled_design_file_path, @rmap, @j_jxds.new_with_sig('Lorg.w3c.dom.Document;Ljava.lang.String;', @j_jrxu.parse(@xis), xpath_filter))
  @j_jem.exportReportToPdf(@jp)
end

def initialize
  #@j_jre = Rjb::import('net.sf.jasperreports.engine.JRException')
  #@j_jp = Rjb::import('net.sf.jasperreports.engine.JasperPrint')

  @j_jem = Rjb::import('net.sf.jasperreports.engine.JasperExportManager')
  @j_jfm = Rjb::import('net.sf.jasperreports.engine.JasperFillManager')
  @j_jxds = Rjb::import('net.sf.jasperreports.engine.data.JRXmlDataSource')
  @j_sxis = Rjb::import('org.xml.sax.InputSource')
  @j_jrxu = Rjb::import('net.sf.jasperreports.engine.util.JRXmlUtils')
  @j_iosr = Rjb::import('java.io.StringReader')
  @rmap = Rjb::import('java.util.HashMap').new
  @xis = @j_sxis.new()
end
end
end

Note: I am assuming that the JVM is loaded once. I don't know about GC and the Rjb::import objects, it seems that they are ruby proxy stubs so I guess they will get garbage collected as normal. I tried the Rjb::unload method in the local trial but it seg faulted the ruby interpreter, I did not try it on Rails.

This technique opens up using other Java libraries.

Excel Exporting: Multiple Worksheets

Right... onto the first nugget. How does one create multiple worksheet workbook export from Rails?

My solution was to save a sample as xml using Excel, then to break it up into chunks of static partials, a bit like this.
- workbook start
- worksheet start
- worksheet end
- worksheet start
- worksheet end
- workbook end

I chose to combine workbook start with worksheet start, worksheet end with worksheet start and worksheet end with workbook end

I created quite a few .rxml files that internally looked like this...

xml << %Q¬
<?xml version="1.0"?><?mso-application progid="Excel.Sheet"?>
<Workbook xmlns="urn:schemas-microsoft-com:office:spreadsheet" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:html="http://www.w3.org/TR/REC-html40">
<DocumentProperties xmlns="urn:schemas-microsoft-com:office:office">
<Author>author</Author> <Company>Company</Company>
<Version>11.8107</Version>
</DocumentProperties>
  <ExcelWorkbook xmlns="urn:schemas-microsoft-com:office:excel">
  <WindowHeight>7545</WindowHeight>
  <WindowWidth>16260</WindowWidth>
  <WindowTopX>165</WindowTopX>
  <WindowTopY>105</WindowTopY>
  <ActiveSheet>2</ActiveSheet>
  <ProtectStructure>False</ProtectStructure>
  <ProtectWindows>False</ProtectWindows>
</ExcelWorkbook>
<Styles>
  <Style ss:ID="Default" ss:Name="Normal"> <Alignment ss:Vertical="Bottom"/> <Borders/> <Font/> <Interior/> <NumberFormat/> <Protection/>
 </Style>
</Styles>
<Worksheet ss:Name="Sheet1">
¬

NOTE: There are more Excel Style sections than shown above.
I called this partial _wb_start.rxml.

If all of your worksheets share common header rows you can create another partial with this column data in it, like so...

xml << %Q¬
  <Column ss:AutoFitWidth="0" ss:Width="100.5"/>
  <Column ss:AutoFitWidth="0" ss:Width="65.5"/>
  <Column ss:AutoFitWidth="0" ss:Width="134.25"/>
  <Column ss:AutoFitWidth="0" ss:Width="134.25"/>
  <Column ss:AutoFitWidth="0" ss:Width="105"/>
  <Column ss:AutoFitWidth="0" ss:Width="105"/>
  <Column ss:AutoFitWidth="0" ss:Width="28.5"/>
  <Column ss:AutoFitWidth="0" ss:Width="251.5"/>
  <Column ss:AutoFitWidth="0" ss:Width="63.75"/>
  <Column ss:AutoFitWidth="0" ss:Width="15.5"/>
  <Column ss:AutoFitWidth="0" ss:Width="80.5"/>
  <Column ss:AutoFitWidth="0" ss:Width="80.5"/>
  <Column ss:AutoFitWidth="0" ss:Width="465.5"/>
  <Row ss:AutoFitHeight="0" ss:Height="18" ss:StyleID="s24">
    <Cell ss:StyleID="s22"/>
    <Cell ss:StyleID="s23"/>
    <Cell ss:StyleID="s23"/>
    <Cell ss:StyleID="s23"/>
    <Cell ss:StyleID="s23"/>
    <Cell ss:StyleID="s23"/>
  </Row>
  <Row>
    <Cell ss:StyleID="s42"><Data ss:Type="String">Col head 01</Data></Cell>
    <Cell ss:StyleID="s42"><Data ss:Type="String">Col head 02</Data></Cell>
    <Cell ss:StyleID="s42"><Data ss:Type="String">Col head 03</Data></Cell>
    <Cell ss:StyleID="s43"><Data ss:Type="String">Col head 04</Data></Cell>
    <Cell ss:StyleID="s42"><Data ss:Type="String">Col head 05</Data></Cell>
    <Cell ss:StyleID="s42"><Data ss:Type="String">Col head 06</Data></Cell>
    <Cell ss:StyleID="s42"><Data ss:Type="String">Col head 07</Data></Cell>
    <Cell ss:StyleID="s42"><Data ss:Type="String">Col head 08</Data></Cell>
    <Cell ss:StyleID="s43"><Data ss:Type="String">Col head 09</Data></Cell>
    <Cell ss:StyleID="s45"/>
    <Cell ss:StyleID="s46"><Data ss:Type="String">Col head 10</Data></Cell>
    <Cell ss:StyleID="s46"><Data ss:Type="String">Col head 11</Data></Cell>
    <Cell ss:StyleID="s46"><Data ss:Type="String">Col head 12</Data></Cell>
  </Row>
¬

You would then include this partial for each worksheet, details later.

Then you need to create another partial which ends one worksheet and starts the next, or you could have a generic end-worksheet partial. Keep doing this until you have a bunch to partials which describe the start and end of the workbook and sheets.
Here is an example of a worksheet end with a worksheet start

xml << %Q¬
  <WorksheetOptions xmlns="urn:schemas-microsoft-com:office:excel">
   <PageSetup>
    <Header x:Margin="0.51181102362204722"/>
    <Footer x:Margin="0.51181102362204722"/>
    <PageMargins x:Bottom="0.59055118110236227" x:Left="0.74803149606299213"
     x:Right="0.55118110236220474" x:Top="0.59055118110236227"/>
   </PageSetup>
   <Print>
    <ValidPrinterInfo/>
    <PaperSizeIndex>9</PaperSizeIndex>
    <HorizontalResolution>600</HorizontalResolution>
    <VerticalResolution>0</VerticalResolution>
   </Print>
   <Panes>
    <Pane>
     <Number>3</Number>
     <ActiveRow>36</ActiveRow>
     <ActiveCol>1</ActiveCol>
    </Pane>
   </Panes>
   <ProtectObjects>False</ProtectObjects>
   <ProtectScenarios>False</ProtectScenarios>
  </WorksheetOptions>
  <Sorting xmlns="urn:schemas-microsoft-com:office:excel">
   <Sort>Part Number</Sort>
  </Sorting>
 </Worksheet>
 <Worksheet ss:Name="RMA Form">
¬

I then created a rxml file which is rendered from the action in the controller.

xml << render(:partial => 'wb_start_ws1_start')
xml.Table do
  xml << render(:partial => 'ws_headings')
  #dynamic data here
end #table
xml << render(:partial => 'ws1_end_ws2_start')
xml.Table do
  xml << render(:partial => 'ws_headings')
  #dynamic data here
end # table
xml << render(:partial => 'ws2_end_ws3_start')
xml.Table do
  xml << render(:partial => 'ws_headings')
  #dynamic data here
end # table
xml << render(:partial => 'ws3_end_wb_end')

Note the bit that goes: xml << render(:partial => 'wb_start')
This is how you render the static xml into the dynamic Builder output at certain points.
The #dynamic data bit looks like this

  aattrs = [:fld1,:fld2,:fld3,:fld4]
  atypes = ['String','String','Number','String']
  len = aattrs.length - 1
  @collection.each do obj
    xml.Row do
      (0..len).each do i
        xml.Cell 'ss:StyleID'=>"s76" do
          dat = obj.send(aattrs[i])
          if dat
            xml.Data dat, 'ss:Type' => atypes[i]
          else
            xml.Data "#N/A", 'ss:Type' => 'String'
          end
        end
      end
      # add extra cells to pad out the styles
      xml.Cell 'ss:StyleID'=>"s45"
      3.times do
        xml.Cell 'ss:StyleID'=>"s46" #nice bordered light blue boxes
      end
    end

So there you have it. You can produce Excel XML multiple worksheet workbook exports from Rails with exactly the styles and number formats as a regular Excel workbook.
Far better than CSV.