Recent Blog Entries

Results-Only Interface with Quest via SOAP / HL7

<Rant>

Just as a note, I think SOAP is a terrible tragedy. There are dramatically better options. Check out Designing Hypermedia APIs by Steve Klabnik.

</Rant>

Get Results ( HL7 w/ Embedded PDF )

I’m using the action ‘get_results’ with parameters like this:

1
2
3
4
5
6
7
8
     'resultsRequest' =>
          {
            'startDate' => self.start_date.to_date.to_s(:mdy),
            'endDate' => self.end_date.to_date.to_s(:mdy),
            'maxMessages' => 30,
            'providerAccounts' => 'THO',
            'retrieveFinalsOnly' => 'false'
          }

and the request looks like this:

1
2
3
4
5
6
SOAP request: https://cert.hub.care360.com/resultsHub/observations/hl7
SOAPAction: "getResults"
Content-Type: text/xml;charset=UTF-8
Content-Length: 828

<?xml version="1.0" encoding="UTF-8"?><env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsdl="http://medplus.com/resultsHub/observations" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ins0="java:com.medplus.serviceHub.results.webservice" xmlns:ins1="java:com.medplus.serviceHub.results.webservice.printable" xmlns:ins2="java:javax.xml.rpc" xmlns:ins3="java:javax.xml.soap" xmlns:ins4="java:language_builtins.lang"><env:Body><getResults><wsdl:resultsRequest><wsdl:startDate>08/01/2013</wsdl:startDate><wsdl:endDate>09/01/2013</wsdl:endDate><wsdl:maxMessages>30</wsdl:maxMessages><wsdl:providerAccounts>THO</wsdl:providerAccounts><wsdl:retrieveFinalsOnly>false</wsdl:retrieveFinalsOnly></wsdl:resultsRequest></getResults></env:Body></env:Envelope>

response looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
<?xml version="1.0"?>
<env:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <env:Header/>
  <env:Body env:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
    <m:getResultsResponse xmlns:m="http://medplus.com/resultsHub/observations">
      <result xmlns:n1="java:com.medplus.serviceHub.results.webservice" xsi:type="n1:ResultsResponse">
        <HL7Messages soapenc:arrayType="xsd:string[2]">
          <string xsi:type="xsd:string">MSH|^~\&amp;|LAB|QTE||12345|20130404215632||ORU^R01|80000000000000053080|D|2.3.1
PID|1|PID13|CB045678A||TEST^TC13||19700615|F|||||^^^^^972^9163000|||||0404135|111220001
NTE|1|TX|TEST CASE 13
ORC|RE||CB018665A||CM|||||||1122334455^ALLEN^JOSEPH^^^^^^^^^^NPI
OBR|1||CB018665A|3020^URINALYSIS, COMPLETE W/REFLEX TO CULTURE^^3020SBX=^URINALYSIS, COMPLETE W/REFLEX TO CULTURE|||20130404093200|||||||20130404083500||1122334455^ALLEN^JOSEPH^^^^^^^^^^NPI|||||CB^Quest Diagnostics-Wood Dale^1355 Mittel Blvd^Wood Dale^IL^60191-1024^Anthony V Thomas, M.D.|20130805171655|||F
OBX|1|ST|5778-6^Color Ur^LN^30005500^COLOR^QDIWDL||AMBER||YELLOW|A|||F|||20130805171655|CB
OBX|2|ST|5767-9^Appearance Ur^LN^30005600^APPEARANCE^QDIWDL||TURBID||CLEAR|A|||F|||20130805171655|CB
OBX|3|NM|5811-5^Sp Gr Ur Strip^LN^30006000^SPECIFIC GRAVITY^QDIWDL||1.032||1.001-1.035|N|||F|||20130805171655|CB
OBX|4|NM|5803-2^pH Ur Strip^LN^30006200^PH^QDIWDL||7.8||5.0-8.0|N|||F|||20130805171655|CB

<snip />

OBR|9||CB018665A|ClinicalPDFReport1^Clinical PDF Report CB018665A-1^^ClinicalPDFReport1^Clinical PDF Report CB018665A-1|||20130404093200|||||||20130404083500||1122334455^ALLEN^JOSEPH^^^^^^^^^^NPI||||||20130805171655|||F
OBX|1|ED|ClinicalPDFReport1^Clinical PDF Report CB018665A-1^^ClinicalPDFReport1^Clinical PDF Report CB018665A-1||QTE^Image^PDF^Base64^JVBERi0xLjQKJeLjz9MKMyAwIG9iago8PC9GaWx0ZXIvRmxhdGVEZWNvZGUvTGVuZ3RoIDEwPj5zdHJlYW0KeJwr5AIAAO4AfAplbmRzdHJlYW0KZW5kb2JqCjQgMCBvYmoKPDwvRmlsdGVyL0ZsYXRlRGVjb2RlL0xlbmd0aCA2Mz4+c3RyZWFtCnicUwjkKuRyCuHSj8g0VLBUCEnjMlQwAEJDBVNjIwVjA4WQXC6NAEd3VwUjBX83BSPNkCwu1xCuQC4AVXQLzApl

<snip content="large chunks of base64 encoded PDF" />

l0+PnN0cmVhbQp4nGNgYPj/n4mBiYGBUfEKkGDgBxHRDAwgMUZGhtsQFieIYGYUYYJwWUAEK4hgAxHsIIIDRDAxck4FGqCwGUQ8ZIAAAKN5Bx4KZW5kc3RyZWFtCmVuZG9iagpzdGFydHhyZWYKOTE0MAolJUVPRgo=||||||F
</string>
        </HL7Messages>
        <isMore xsi:type="xsd:boolean">false</isMore>
        <requestId xsi:type="xsd:string">551514a50a801e1512b2a98ed5f30b36</requestId>
      </result>
    </m:getResultsResponse>
  </env:Body>
</env:Envelope>

Acknowledgement

Using the response document above, I need the request ID and the message control IDs. The request ID comes from the SOAP document. The message control IDs come from the HL7 MSH segment.

1
2
>> response_document.xpath('//requestId').text
=> "551514a50a801e1512b2a98ed5f30b36"

And here’s what I send to Quest…

1
2
3
4
5
6
7
8
9
SOAP request: https://cert.hub.care360.com/resultsHub/observations/hl7
SOAPAction: "acknowledgeResults"
Content-Type: text/xml;charset=UTF-8
Content-Length: 1029


<?xml version="1.0" encoding="UTF-8"?><env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsdl="http://medplus.com/resultsHub/observations" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ins0="java:com.medplus.serviceHub.results.webservice" xmlns:ins1="java:com.medplus.serviceHub.results.webservice.printable" xmlns:ins2="java:javax.xml.rpc" xmlns:ins3="java:javax.xml.soap" xmlns:ins4="java:language_builtins.lang"><env:Body><acknowledgeResults><wsdl:requestId>552173510a801e1512b2a98ee228b506</wsdl:requestId><wsdl:acknowledgeMessages><wsdl:message>MSH|^~&amp;|PhysioAge Reporting|12345|LAB|QTE|201310091854||ACK|1372089260531|D|2.3.1
MSA|CA|80000000000000053080</wsdl:message></wsdl:acknowledgeMessages><wsdl:acknowledgeMessages><wsdl:message>MSH|^~&amp;|PhysioAge Reporting|12345|LAB|QTE|201310091854||ACK|1372089260531|D|2.3.1
MSA|CA|80000000000000059438</wsdl:message></wsdl:acknowledgeMessages></acknowledgeResults></env:Body></env:Envelope>

and I get back:

1
<env:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><env:Header></env:Header><env:Body env:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><m:acknowledgeResultsResponse xmlns:m="http://medplus.com/resultsHub/observations"></m:acknowledgeResultsResponse></env:Body></env:Envelope>

Search w/ Dynamic Options via collection+json

We’re searching for people using ‘location’, ‘radius’, and ‘specialization’ fields. Since collection+JSON apparently doesn’t provide a mechanism for delivering options like the HTML tag `<option>`, we’ve forked the Ruby gem for collection+JSON and added that feature:

[Find our repo with details at GitHub]https://github.com/mustmodify/collection-json.rb

Given I am an anonymous user
When I go to the search page
and I enter my postal code
And I select a specialty
And I submit the form
Then I should be on the results page
And I should see my criteria in an updatable form
And I should see the results

And another important scenario:

Given I am on the result page
Then I should see my search criteria
And I should see a list of the cities where the resulting people are located
And I should be able to filter by those cities.

Because of the second scenario, we need to pass back not just the results, but also the inquiry details, which will include our criteria and city list. Collection+JSON is implicitly limited to one kind of result per response, ie you can’t have a collection with both an inquiry item AND a result item, since there is no rel-tag in the item. And even if there were a rel-tag… having two kinds of data mixed up together would be a giant mess.

So our challenge is to present meta-data about the search, available filters, and the results.

Our modified collection+json includes a ‘related’ collection which could hold the inquiry information. However, we need the expressiveness of the ‘template’ section for this task. So the ‘items’ collection links to the inquiry that has been submitted. items0.links gives you a link to the results.

Our modified collection+JSON schema includes ‘options’ in the template/query section. This is the perfect mechanism for sending back the list of cities.

But that still leaves the problem of meta-data…. how many results were there? What page are we on? etc.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
{
    "collection": {
        "href": "/inquiries/4.json",
        "items": [
            {
                "href": "/inquiries/4.json",
                "data": [
                    {
                        "name": "location",
                        "value": "70508"
                    },
                    {
                        "name": "radius"
                    },
                    {
                        "name": "specialization_id",
                        "value": "12"
                    }
                ],
                "links": [
                    {
                        "href": "/inquiries/4/results.json",
                        "rel": "specialist_search_results_resource"
                    }
                ]
            }
        ],
        "template": {
            "data": [
                {
                    "name": "inquiry[location]",
                    "prompt": "Location",
                    "value": "70508"
                },
                {
                    "name": "inquiry[search_area]",
                    "prompt": "Where are you looking?",
                    "options": [
                        {
                            "value": "local",
                            "prompt": "Close to Me"
                        },
                        {
                            "value": "us",
                            "prompt": "Anywhere in the US"
                        },
                        {
                            "value": "global",
                            "prompt": "Anywhere in the world."
                        }
                    ]
                },
                {
                    "name": "inquiry[specialization_id]",
                    "prompt": "Category",
                    "value": 12,
                    "options": [
                        {
                            "value": 1,
                            "prompt": "Rails"
                        },
                        {
                            "value": 2,
                            "prompt": "Ruby"
                        },
                        {
                            "value": 3,
                            "prompt": "AngularJS"
                        },
                        {
                            "value": 4,
                            "prompt": "jQuery"
                        }
                    ]
                }
            ]
        }
    }
}

In order to do this, we will need to have two kinds of locations… a home location and a cities collection. Our cities are actually [CBSAs]:http://en.wikipedia.org/wiki/Core_Based_Statistical_Area, so I’ll call them Areas.

Since no values selected would necessarily mean no results, we’ll assume a null value means all values. That way, the initial submit will come back something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
                {
                    "name": "inquiry[home]",
                    "prompt": "I live in",
                    "value": "70508",
                },
                {
                    "name": "inquiry[areas]",
                    "prompt": "Locations",
                    "value": [ 10100, 31080, 31060, 35440 ],
                    "options": [
                        {
                            "value": 10100,
                            "prompt": "Aberdeen, SD"
                        },
                        {
                            "value": 31080,
                            "prompt": "Los Angeles"
                        },
                        {
                            "value": 31060,
                            "prompt": "Los Alamos, NM"
                        },
                        {
                            "value": 35440,
                            "prompt": "Newport, OR"
                        }
                    ]
                }

Update

We’ve added a ‘meta’ element. See discussion here: https://groups.google.com/forum/#!topic/collectionjson/K2ZibVKpA6Q

reverse lookup which package caused a file to exist

1
2
jw@logopolis:~$ sudo dpkg -S /lib/init/vars.sh
initscripts: /lib/init/vars.sh

cool.

/hattip glitsj16 on #ubuntu

sending parameters with brackets [ ] via AngularJS

I want an AngularJS controller to POST to a Rails endpoint with parameters like { ‘item[price]’: ‘32.50’ } and have Rails see it as “item” => {"price": “32.50”}. This works perfectly well in normal HTML… it’s not clear why it isn’t working in Angular.

Via HTTP

It works in a normal HTML form:

1
2
3
4
<form action="/items.json" method="POST">
  <input name="item[price]" type="text" value="32.50" />
  <input type="submit">Save</input>
</form>

submitting this form, we see this from Rails:

1
2
3
Started POST "/items.json" for 192.168.1.20 at 2013-12-06 09:47:04 -0600
Processing by ItemsController#create as JSON
  Parameters: {"item"=>{"price"=>"32.50"}}

via $http.post

1
$http.post('/items.json', {'item[price]': '32.50'});

and Rails sees:

1
2
3
Started POST "/items.json" for 192.168.1.20 at 2013-12-06 09:50:39 -0600
Processing by ItemsController#create as JSON
  Parameters: {"item[price]"=>"32.50", "item"=>{"item[price]"=>"32.50"}}

So what’s different?

Checking Chrome’s developer tools, under the network tab > Headers, we get some answers

When using a form, I see a section labeled “Form Data”. After clicking “View Source” I get this:

item%5Bprice%5D=32.50

When using AngularJS, I see a section labeled “Request Payload”. After clicking “View Source”, I get this:

{"item[price]“:”32.50"}

AH HA!

Final Solution

1
2
3
4
5
6
7
8
9
$scope.submit_data = function()
{
    var xsrf = $.param({'item[price]': '32.50'});

    $http.post('/items.json',
      xsrf,
      {headers: {'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8'}}
    );
}

References

What is the difference between form data and request payload?

How can I make angular.js post data as form data instead of a request payload?

Evolving thoughts about JSON APIs 2: a reason for collection+json

This is a follow-up to http://blog.mustmodify.com/2013/10/18/evolving-thoughts-about-apis

cJ is short for collection+JSON because lazy.

Reasons for cJ

Although I normally develop full-stack applications, I was contracted to implement a new API using a somewhat consistent pattern and was getting some positive feedback about it from the dev who was using it to build a site. (Yes, strange not to be doing full-stack. Not my decision.)

I remembered an inspiring presentation by Ruby Hero Steve Klabnik during RubyConf 2012 (?) . The focus of his talk was that APIs should present basically the same information as browsers… and that API clients should consume APIs in basically the same way as browsers… which is to say that they wouldn’t “just know” paths on a system. They would go to the welcome page and be able to discover hyperlinks to … basically the rest of the system. Just like on the webs. And if a link were to change, that would be OK because the welcome page’s link would change. The only commitment would be a list of ‘rel’ tags… a list of tags that provide context to the page.

Although I can’t find the original presentation, he obviously has given this talk many times. I found one on the web. He talked about a few API protocols / mime-types… after looking at them, some seemed awkward, others immature, and others overly-erudite. So I figured I could roll my own. lolz. Long story short, I think what I had was pretty decent, but the front-end dev said that inconsistencies between the ‘instance’ views ( /items/1.json ), the ‘resource’ views ( /items.json ) and the ‘welcome’ page ( /index.json ) were a bit of a pain point. Not a big deal, but something that needed handling. And wouldn’t it be nice if it were consistent? But it can’t be, because on the welcome page, there is no “object” it’s just… welcome. Here’s what you can do. And on the resource page, it’s just one resource. Collection page, collections.

That reminded me about collection+json. So we’re going to give it a shot.

Technical Stack

I started out using jBuilder. Unfortunately, jBuilder doesn’t lend itself to building collections. And as the name implies, that happens alot in collection+JSON. So I ended up doing hackish things like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
link_collection = [
  {
    :href => items_path,
     :rel => 'parent'
  },
  {
    :href => manufacturers_path,
    :rel => 'manufacturer_resource'
  }
]

@item.options.each do |option|
    link_collection << 
    {
      :href => options_path( option ),
      :rel => 'option_details'
    }
end

json.array! link_collection do |link|
  json.link do |json|
    json.href link[:href]
    json.rel link[:rel]
  end
end

it’s not the worst code ever, but it’s awkward. I ended up moving the link-building code to a helper just because I didn’t want it in the view, but then… it was somewhere else. Anyway, suboptimal.

We switched to the collection-json gem ( NOTE that this is not the same as the collection_json gem, which is apparently no longer maintained? ) and the same code looks more like this:

1
2
3
4
5
  api.add_link category_path( item.category.first ), 'category'
  api.add_link manufacturer_path( item.manufacturer ), 'manufacturer'
  @item.options.each do |option|
    api.add_link option_path( option ), 'option_details'
  end

It’s a lot better. So far I’ve been modeling each endpoint in app/endpoints…

1
2
3
4
5
6
7
class ItemsEndpoint < API
  def to_json(atts = {})
    CollectionJSON.generate_for(context.request.path) do |api|
      api.add_item( whatever )
    end
  end
end

and app/endpoints/api.rb:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
class API < Valuable
  has_value :context
  has_value :collection
  has_value :singleton, :klass => :boolean

  def current_user
    context.send(:current_user)
  end

  def method_missing(method, *args)
    if context.respond_to?(method)
      context.send(method, *args)
    else
      super
    end
  end
end

I use the instance flag to determine whether to show nested resources at collection.items0.links or collection.links. I’m looking at making some minor changes moving forward, but this has worked well.

from the controller:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
  # GET /items
  # GET /items.json
  def index
    @items = Item.visible_to( current_user )  # visible_to is an AR scope

    respond_to do |format|
      format.html do
        @items = @items.paginate(:page => params[:page])
        # though sometimes I do pagination in json, too. Hopefully I'll get around to posting about that.
      end

      format.json { render :json => ItemsEndpoint.new(:context => view_context, :collection => @items).to_json }
    end
  end

  # GET /items/1
  # GET /items/1.json
  def show
    @item = Item.visible_to(current_user).where(:id => params[:id]).first!

    respond_to do |format|
      format.html {}
      format.json { render :json => ItemsEndpoint.new(:context => view_context, :collection => [@item], :singleton => true ).to_json }
    end
  end

  ... and much much more.

installing rubinius + puma on EC2 with Amazon Linux

I have been struggling to get this to work, so I thought I would start fresh and write down the process in order to avoid making random decisions.

Overview

  1. install rubinius
  2. install bundler
  3. install MySQL
  4. cap deploy:setup
  5. cap deploy
  6. puma . # see it work at all
  7. get puma to run as a service

Install Rubinius

options:

  • install MRI and then install Rubinius
  • install Rubinius via rvm, chruby, rbenv, etc.

I use rvm in dev. I honestly don’t know the implications of running rubinius / puma via rvm, chruby, etc., and was not able to get any useful information from #rubinius. For now, since I’m comfortable with it, I’m going with rvm.

get puma to run as a service

wget https://raw.github.com/puma/puma/master/tools/jungle/init.d/puma

References

Gist: ‘Puma + Nginx + Capistrano’
“Puma as a Service”:

Evolving thoughts about building APIs

Ever since I saw him speak about the subject at RubyConf 2011, I have been fascinated by Steve Klabnik’s ideas about Hypermedia APIs. I have been keeping up with his evolving book on the subject, Designing Hypermedia APIs and have enjoyed it.

A new client is asking me to design an API. I started with just the basic REST stuff. Having finished 12 resources, I went to work on the “welcome” page. Steve said that an API should be “fully discoverable”, and I think this concept is great. Basically, a client should be able to go to a site’s index page, look for a certain link (just as a human would with a web browser) and then be taken to the relevant resource. If the URLs change around, that shouldn’t matter. The API should facilitate that.

So far, my convention has been to include EITHER links to nested resources or to embed nested resources, and include a ‘resource’ link… so for a building, it might look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
{
  building:
  {
    resource: '/buildings/3241.json',
    tenants:
    [
      { 
        resource: '/tenants/32342.json'
        name: 'Praeses LLC',
        floors: [4, 5, 8],
        reception: 410,
        lease-resource: '/tenants/32342/lease.json'
      },
      {
        resource: '/tenants/32341.json',
        name: 'Dewey, Cheatum & Howe, Attorneys at Law',
        floor: [7, 8],
        reception: '710',
        lease-resource: '/tenants/32341/lease.json'
      }
    ],
    custodians-resource: '/buildings/3241/custodians.json',
    administrativa-resource: '/buildings/3241/administrators.json',
  }
}

so my initial page, the one that has the links to all of the available resources, now looks like this:

1
2
3
4
5
6
{
  buildings-resource: '/buildings.json',
  tenant-profile-resource: /tenants/3234.json',
  building-search-resource: '/buildings/search.json',
  user-session-resource: '/user-session.json'
}

this is fine, I guess. having a search as a resource might phase some people but I have always looked at search as a resource, so I’m cool with it.

I’m now re-visiting Klabnik’s book and also a video of him presenting at a conference in LA. He recommends the following mime-type: collection+json, Cj for short.

The same initial page would look like this in Cj:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
{
    "collection": {
        "href": "/",
        "items": [
            {
                "href": "/users/21001.json",
                "data": {
                    "email": "your-email@some.test"
                }
            },
        ],
        "links": [
            {
                "name": "tenant",
                "href": "/tenants/3234.json",
                "rel": "profile"
            },
            {
                "name": "tenants",
                "href": "/tenants.json",
                "rel": "resource"
            },
            {
                "name": "buildings",
                "href": "/buildings.json",
                "rel": "resource"
            },
            {
                "name": "user-session",
                "href": "/user_session.json",
                "rel": "resource"
            }
        ],
        "queries": [
            {
                "name": "buildings-search",
                "href": "/buildings/search.json",
                "rel": "search",
                "data": [
                    {
                        "name": "q",
                        "value": ""
                    },
                    {
                        "name": "postal_code",
                        "value": ""
                    }
                ]
            }
        ]
    }
}

The ‘items’ element should be the result of the current resource… it isn’t clear to me whether the current resource is the user, the tenant profile, or the user session… but obviously that would be much more clear for most of the pages.

Some Confusion

The thing that most confuses me here is the use of ‘name’ and ‘rel’ for the links. Should I put {name: ‘tenant’, rel: ‘profile’, href: ‘/tenants/3234.json’} and also {name: ‘tenant’, rel: ‘resource’, href: ‘/tenants.json’} ? or is rel additional data where name can be used to find the needed resource? Or should rel be the primary key for finding specific links?

I have heard that the ‘rel’ tag should describe how the link relates to the current resource. IIRC, I heard someone else suggest that you have a list of rel tags. Something like:

  • tenants: the tenant collection resource.
    GET => list, POST => create
  • new-tenant: gives you the fields needed to create a tenant GET => read
  • tenant: a specific tenant’s resource. GET => read, POST / PATCH => update, DELETE => destroy
  • tenant-profile: if the current user is associated with a specific tenant, this is a link to that tenant’s profile
  • user-profile: resource for the current user
  • users: user collection resource
  • user
  • building-search: allows you to create searches and view results. GET/POST => search. Having search be its own resource allows you to do interesting things like caching search results, and having a page that will return the input fields
  • new-building-search: the page that would return the search fields. For instance, q and zipcode.

Decision Time

So this seems like a lot more fun. I’m about to deploy this app and presumably someone will start building the front-end, so this is the decision point-of-no-return… should I stick with what I have, or start using something like collection+json?

My big question at this point is whether it solves a problem I anticipate having with my current process.

Cj feature can I do it now?
list the current resource yes
links to related resources yes
links to parent or unrelated resources unknown
form parameters yes
form options a-la a drop down no update: now that I think about it, I can sorta do it. Not as neatly, though.
rel tags no
errors yes

Although this is a well-thought-out hypermedia mime type, my impression is that it might add some complexity for the client. Complexity is fine, as long as there is a benefit. SOAP, for instance, adds complexity but, in my experience, doesn’t provide any benefit over REST in exchange for the complexity.

Looking at this, the only thing that worries me is not having rel tags… though I can’t think of an example where that would be a problem. Conclusion: stick with what I have.

References

Designing Hypermedia APIs
Collection+JSON Primer
Collection+JSON support in Roar!

Attempting to acknowledge Quest Diagnostics results

I am successfully retrieving HL7 messages, including data and embedded PDF, via Ruby. I wish I had kept better notes… so I’m doing that now. My current challenge is to send an ACK message.

I can tell that the people at Quest were very satisfied with their recent guidance to me, which I can fairly summarize as:

Oh, you were using our NEW API. No one is using that yet. Go back to the one labeled “legacy.”

Very enterprisey in its own way.

The endpoint I am using to retrieve results is:

https://cert.hub.care360.com:443/resultsHub/observations/hl7

The documentation describes this endpoint as:
This is to retrieve HL7 ORU messages only from the HUB, this would also include embedded pdfs in the HL7 ORU message

Do not attempt to make sense of the fact that it says we provide only x, and also y and z.

So the WSDL file I have includes the following actions:

1
2
3
      >> Interfaces::Quest.new.soap_interface.client.wsdl.soap_actions

      => [:get_results, :get_more_results, :acknowledge_results, :get_hl7_results, :get_more_hl7_results, :acknowledge_hl7_results, :get_provider_accounts]

I am using ‘get_results’ ( which presumably would translate to GetResults in the actual WSDL.) Looking at the parameters I’m sending and the WSDL file, it seems like I need to include any ComplexType names as a key with the parameters of that complex type being in a hash.

Since I’m using ‘get_results’ to get results, I’m going to try using ‘acknowledge_results’ to … acknowledge… the … results.

Here’s the WSDL:

1
2
3
4
5
6
7
8
9
10
     <message   name="acknowledgeResults">
      <part    xmlns:partns="http://www.w3.org/2001/XMLSchema"
        type="partns:string"
        name="requestId">
      </part>
      <part    xmlns:partns="java:language_builtins.lang"
        type="partns:ArrayOfString"
        name="acknowledgeMessages">
      </part>
     </message>

no complex types here.

Question: does the Request ID come from Quest’s response to my original SOAP message? Or the ID of the HL7 message I’m acknowledging? Because it would be more complicated, I’m going to assume the first one.

Question There are two parts. Are these supposed to be sent as just two child nodes of a collection? If not, how?

I’m going to their HUB MedPlus HIT toolkit (13.2) for examples… which I will pull from sample_messages/soap/results/legacy/observation/acknowledgeResults.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
      <?xml version="1.0" encoding="UTF-8"?>
      <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
              <soapenv:Body>
                      <ns1:acknowledgeResults soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="http://medplus.com/observation">
                              <ack href="#id0"/>
                      </ns1:acknowledgeResults>
                      <multiRef id="id0" soapenc:root="0" soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xsi:type="ns2:Acknowledgment" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns2="java:com.medplus.serviceHub.results.webservice.observation">
                              <acknowledgedResults soapenc:arrayType="ns2:AcknowledgedResult[1]" xsi:type="soapenc:Array">
                                      <acknowledgedResults href="#id1"/>
                                      </acknowledgedResults>
                              <requestId xsi:type="xsd:string">bd078d98c0a83801008397d28f1918ca</requestId>
                      </multiRef>
                      <multiRef id="id1" soapenc:root="0" soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xsi:type="ns3:AcknowledgedResult" xmlns:ns3="java:com.medplus.serviceHub.results.webservice.observation" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/">
                              <ackCode xsi:type="xsd:string">ACK</ackCode>
                              <documentIds soapenc:arrayType="xsd:string[1]" xsi:type="soapenc:Array">
                                      <documentIds xsi:type="xsd:string">19784</documentIds>
                              </documentIds>
                              <rejectionReason xsi:type="xsd:string" xsi:nil="true"/>
                              <resultId xsi:type="xsd:string">bd078f8cc0a8380100c6235dadf0c33e</resultId>
                      </multiRef>
              </soapenv:Body>
      </soapenv:Envelope>

SOAP envelopes and whatever-else aside, the structure seems to be:

*acknoledgeResults

  • ack href=“something”
  • multiRef
    • acknowledgedResults
      ***acknowledgedResults href
    • requestId
  • multiRef
    • acknowledgedResults
      ***acknowledgedResults href
    • requestId
      *multiRef
    • ackCode ACK
    • documentIds
      • documentIds 19784
    • rejectionReason nil
    • resultId bd078f8cc0a8380100c6235dadf0c33e

    Well, that XML doesn’t match up with the ‘parts’ from the WSDL at all, so … that must be the wrong example file.

    So now I’ll try sample messages\soap\results\observation:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    
          <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:res="http://medplus.com/results">
             <soapenv:Header/>
             <soapenv:Body>
                <res:acknowledgeResults>
                   <res:RetrieveResultsAcknowledge>
                      <ackMessages>
                         <controlId>ca3ce8a6ac1262891d36757221fb4e7b</controlId>
                         <message>TVNIfF5+XCZ8fEJyZW50IFByb3ZpZGVyIEFjY291bnR8TEFCfEJyZW50IFByb3ZpZGVyfDIwMTMwMjExMDAwMDAwLjAwMDAtMDcwMHx8QUNLfGJzMjV8RHwyLjMNTVNBfENBfGJzMjV8U1VDQ0VTU0ZVTA0=</message>
                      </ackMessages>
                      <ackMessages>
                         <controlId>ca3ce8b3ac1262891d36892e9c4b5be3</controlId>
                         <message>TVNIfF5+XCZ8fEJyZW50IFByb3ZpZGVyIEFjY291bnR8TEFCfEJyZW50IFByb3ZpZGVyfDIwMTMwMjExMDAwMDAwLjAwMDAtMDcwMHx8QUNLfGJzMjV8RHwyLjMNTVNBfENBfGJzMjV8U1VDQ0VTU0ZVTA0=</message>
                      </ackMessages>
                      <requestId>ca3ce788ac1262891c8775475f428cf8</requestId>
                      <requestParameters>
                         <parameterName></parameterName>
                         <parameterValue></parameterValue>
                      </requestParameters>
                      <resultServiceType>observation</resultServiceType>
                   </res:RetrieveResultsAcknowledge>
                </res:acknowledgeResults>
             </soapenv:Body>
          </soapenv:Envelope>
    

    This doesn’t match either.

    In the absence of examples that match the documentation, I’m taking a step back. I know I have one endpoint working… resultsRequest. So I’m going to look through the examples for any request that is structured in the way mine is structured.

    • sample messages\soap\results\legacy\observation\getResults.xml is somewhat similar, except that it has multiRef nodes … but looking at it, the multiRef node has an ID which you could plug in above and … presumably… get rid of the multiRef node as you replace the thing above with its contents… and then it would be the same.
    • sample messages\soap\results\legacy\observation\getResults-by-provider-acct is similar but not as good a match.
    • sample messages\soap\results\legacy\observation\getResults-dates seems similar… would need a diff to properly compare them.

    None of the other examples in sample messages\soap\results are even close. The other options under “sample messages” are “demographics” and “orders”, so not relevant to me.

    So now I’m going to reverse-engineer an ACK using sample messages\soap\results\legacy\observation\acknowledgeResults.xml

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    
          <?xml version="2.0" encoding="UTF-8"?>
          <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
                  <soapenv:Body>
                          <ns1:acknowledgeResults soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="http://medplus.com/observation">
                                  <ack href="#id0"/>
                          </ns1:acknowledgeResults>
                          <multiRef id="id0" soapenc:root="0" soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xsi:type="ns2:Acknowledgment" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns2="java:com.medplus.serviceHub.results.webservice.observation">
                                  <acknowledgedResults soapenc:arrayType="ns2:AcknowledgedResult[1]" xsi:type="soapenc:Array">
                                          <acknowledgedResults href="#id1"/>
                                          </acknowledgedResults>
                                  <requestId xsi:type="xsd:string">bd078d98c0a83801008397d28f1918ca</requestId>
                          </multiRef>
                          <multiRef id="id1" soapenc:root="0" soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xsi:type="ns3:AcknowledgedResult" xmlns:ns3="java:com.medplus.serviceHub.results.webservice.observation" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/">
                                  <ackCode xsi:type="xsd:string">ACK</ackCode>
                                  <documentIds soapenc:arrayType="xsd:string[1]" xsi:type="soapenc:Array">
                                          <documentIds xsi:type="xsd:string">19784</documentIds>
                                  </documentIds>
                                  <rejectionReason xsi:type="xsd:string" xsi:nil="true"/>
                                  <resultId xsi:type="xsd:string">bd078f8cc0a8380100c6235dadf0c33e</resultId>
                          </multiRef>
                  </soapenv:Body>
          </soapenv:Envelope>
    

    clearing away the soap headers and after some substitution:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    
    <ns1:acknowledgeResults>
          <ack substitution-node="true">
                <acknowledgedResults>
                       <acknowledgedResults  substitution-node="true">
                                  <ackCode>ACK</ackCode>
                                  <documentIds>
                                          <documentIds>19784</documentIds>
                                  </documentIds>
                                  <rejectionReason/>
                                  <resultId>bd078f8cc0a8380100c6235dadf0c33e</resultId>
                        </acknowledgedResults>
                             <requestId xsi:type="xsd:string">bd078d98c0a83801008397d28f1918ca</requestId>
                 </acknowledgedResults>
          </ack>
    </ns1:acknowledgeResults>
    

    So this is pretty reasonable seeming. There are double acknowledgedResults nodes… one of them was the target of a multiRef substitution so it’s hard to know whether to keep or discard that without someone who stares at these things all day… I’ll just experiment. The other target of a multiRef substitution was the ack node… seems like you would want to keep that one. So one of the instances makes me think you do keep the node, the other makes me think you discard it. We’ll just find out.

    I want to compare my existing parameters to the example I like from the samples:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    
    <ns1:getResults>
            <resultRequest href="#id0"/>
                    <endDate xsi:type="xsd:string" xsi:nil="true"/>
                    <maxMessages href="#id1">5</maxMessages>
                    <providerAccounts xsi:type="ns2:ProviderAccount" xsi:nil="true"/>
                    <retrieveFinalsOnly href="#id3">false</retrieveFinalsOnly>
                    <startDate xsi:type="xsd:string" xsi:nil="true"/>
            </resultRequest>
    </ns1:getResults>
    

    compare this with the function I use to generate parameters:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    
        def param_lambda
          lambda {
            {
              'resultsRequest' =>
              {
                'startDate' => self.start_date.to_date.to_s(:mdy),
                'endDate' => self.end_date.to_date.to_s(:mdy),
                'maxMessages' => 30,
                'providerAccounts' => 'THO',
                'retrieveFinalsOnly' => 'false'
              }
            }
          }
        end
    
        def soap_interface
          Interfaces::SOAP.new(
            :wsdl_location => self.class.wsdl_location,
            :wsdl_action => 'get_results',
            :param_lambda => self.param_lambda,
    
            :username => CONFIG[:quest_username],
            :password => CONFIG[:quest_password]
          )
        end
    

    All that gives me the following request:

    1
    2
    3
    4
    
    SOAP request: https://cert.hub.care360.com/resultsHub/observations/hl7
    SOAPAction: "acknowledgeResults", Content-Type: text/xml;charset=UTF-8, Content-Length: 954
    
    <?xml version="1.0" encoding="UTF-8"?><env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsdl="http://medplus.com/resultsHub/observations" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ins0="java:com.medplus.serviceHub.results.webservice" xmlns:ins1="java:com.medplus.serviceHub.results.webservice.printable" xmlns:ins2="java:javax.xml.rpc" xmlns:ins3="java:javax.xml.soap" xmlns:ins4="java:language_builtins.lang"><env:Body><acknowledgeResults><wsdl:ack><wsdl:acknowlegedResults><wsdl:requestId>bd078d98c0a83801008397d28f1918ca</wsdl:requestId><wsdl:acknowledgedResults><wsdl:ackCode>ACK</wsdl:ackCode><wsdl:documentIds>80000000000000026053</wsdl:documentIds><wsdl:rejectionReason xsi:nil="true"/><wsdl:resultId>bd078f8cc0a8380100c6235dadf0c33e</wsdl:resultId></wsdl:acknowledgedResults></wsdl:acknowlegedResults></wsdl:ack></acknowledgeResults></env:Body></env:Envelope>
    

    and response:

    1
    
    Savon::SOAP::Fault: (env:Server) Exception during processing: javax.xml.soap.SOAPException: Found SOAPElement [<wsdl:ack><wsdl:acknowlegedResults><wsdl:requestId>bd078d98c0a83801008397d28f1918ca</wsdl:requestId><wsdl:acknowledgedResults><wsdl:ackCode>ACK</wsdl:ackCode><wsdl:documentIds>80000000000000026053</wsdl:documentIds><wsdl:rejectionReason xsi:nil="true"></wsdl:rejectionReason><wsdl:resultId>bd078f8cc0a8380100c6235dadf0c33e</wsdl:resultId></wsdl:acknowledgedResults></wsdl:acknowlegedResults></wsdl:ack>]. But was not able to find a Part that is registered with this Message which corresponds to this SOAPElement. The name of the element should be one of these[requestId,acknowledgeMessages] (see Fault Detail for stacktrace)
    

    Interesting. So ack > acknowledgeResults has two child elements… requestId and acknoledgedResults. I found some typos above ( the first one that comes to mind is id3/id2 ) so it isn’t a stretch to assume that the inner acknowledgedResults should be acknowledgeMessages.

    New request:

    1
    2
    3
    
    SOAP request: https://cert.hub.care360.com/resultsHub/observations/hl7
    SOAPAction: "acknowledgeResults", Content-Type: text/xml;charset=UTF-8, Content-Length: 954
    <?xml version="1.0" encoding="UTF-8"?><env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsdl="http://medplus.com/resultsHub/observations" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ins0="java:com.medplus.serviceHub.results.webservice" xmlns:ins1="java:com.medplus.serviceHub.results.webservice.printable" xmlns:ins2="java:javax.xml.rpc" xmlns:ins3="java:javax.xml.soap" xmlns:ins4="java:language_builtins.lang"><env:Body><acknowledgeResults><wsdl:ack><wsdl:acknowlegedResults><wsdl:requestId>bd078d98c0a83801008397d28f1918ca</wsdl:requestId><wsdl:acknowledgeMessages><wsdl:ackCode>ACK</wsdl:ackCode><wsdl:documentIds>80000000000000026053</wsdl:documentIds><wsdl:rejectionReason xsi:nil="true"/><wsdl:resultId>bd078f8cc0a8380100c6235dadf0c33e</wsdl:resultId></wsdl:acknowledgeMessages></wsdl:acknowlegedResults></wsdl:ack></acknowledgeResults></env:Body></env:Envelope>
    

    Same error:

    1
    
    Savon::SOAP::Fault: (env:Server) Exception during processing: javax.xml.soap.SOAPException: Found SOAPElement [<wsdl:ack><wsdl:acknowlegedResults><wsdl:requestId>bd078d98c0a83801008397d28f1918ca</wsdl:requestId><wsdl:acknowledgeMessages><wsdl:ackCode>ACK</wsdl:ackCode><wsdl:documentIds>80000000000000026053</wsdl:documentIds><wsdl:rejectionReason xsi:nil="true"></wsdl:rejectionReason><wsdl:resultId>bd078f8cc0a8380100c6235dadf0c33e</wsdl:resultId></wsdl:acknowledgeMessages></wsdl:acknowlegedResults></wsdl:ack>]. But was not able to find a Part that is registered with this Message which corresponds to this SOAPElement. The name of the element should be one of these[requestId,acknowledgeMessages] (see Fault Detail for stacktrace)
    

    so I’ll try getting rid of acknowledgedResults… just … a guess.

    1
    2
    3
    4
    5
    6
    
    SOAP request: https://cert.hub.care360.com/resultsHub/observations/hl7
    SOAPAction: "acknowledgeResults", Content-Type: text/xml;charset=UTF-8, Content-Length: 903
    <?xml version="1.0" encoding="UTF-8"?><env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsdl="http://medplus.com/resultsHub/observations" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ins0="java:com.medplus.serviceHub.results.webservice" xmlns:ins1="java:com.medplus.serviceHub.results.webservice.printable" xmlns:ins2="java:javax.xml.rpc" xmlns:ins3="java:javax.xml.soap" xmlns:ins4="java:language_builtins.lang"><env:Body><acknowledgeResults><wsdl:ack><wsdl:requestId>bd078d98c0a83801008397d28f1918ca</wsdl:requestId><wsdl:acknowledgeMessages><wsdl:ackCode>ACK</wsdl:ackCode><wsdl:documentIds>80000000000000026053</wsdl:documentIds><wsdl:rejectionReason xsi:nil="true"/><wsdl:resultId>bd078f8cc0a8380100c6235dadf0c33e</wsdl:resultId></wsdl:acknowledgeMessages></wsdl:ack></acknowledgeResults></env:Body></env:Envelope>
    HTTPI executes HTTP POST using the httpclient adapter
    SOAP response (status 500):
    Savon::SOAP::Fault: (env:Server) Exception during processing: javax.xml.soap.SOAPException: Found SOAPElement [<wsdl:ack><wsdl:requestId>bd078d98c0a83801008397d28f1918ca</wsdl:requestId><wsdl:acknowledgeMessages><wsdl:ackCode>ACK</wsdl:ackCode><wsdl:documentIds>80000000000000026053</wsdl:documentIds><wsdl:rejectionReason xsi:nil="true"></wsdl:rejectionReason><wsdl:resultId>bd078f8cc0a8380100c6235dadf0c33e</wsdl:resultId></wsdl:acknowledgeMessages></wsdl:ack>]. But was not able to find a Part that is registered with this Message which corresponds to this SOAPElement. The name of the element should be one of these[requestId,acknowledgeMessages] (see Fault Detail for stacktrace)
    

    Try getting rid of the ACK parent node

    1
    2
    3
    4
    5
    6
    7
    8
    
    SOAP request: https://cert.hub.care360.com/resultsHub/observations/hl7
    SOAPAction: "acknowledgeResults", Content-Type: text/xml;charset=UTF-8, Content-Length: 882
    <?xml version="1.0" encoding="UTF-8"?><env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsdl="http://medplus.com/resultsHub/observations" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ins0="java:com.medplus.serviceHub.results.webservice" xmlns:ins1="java:com.medplus.serviceHub.results.webservice.printable" xmlns:ins2="java:javax.xml.rpc" xmlns:ins3="java:javax.xml.soap" xmlns:ins4="java:language_builtins.lang"><env:Body><acknowledgeResults><wsdl:requestId>bd078d98c0a83801008397d28f1918ca</wsdl:requestId><wsdl:acknowledgeMessages><wsdl:ackCode>ACK</wsdl:ackCode><wsdl:documentIds>80000000000000026053</wsdl:documentIds><wsdl:rejectionReason xsi:nil="true"/><wsdl:resultId>bd078f8cc0a8380100c6235dadf0c33e</wsdl:resultId></wsdl:acknowledgeMessages></acknowledgeResults></env:Body></env:Envelope>
    HTTPI executes HTTP POST using the httpclient adapter
    SOAP response (status 500)
    
    
    <env:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><env:Header></env:Header><env:Body><env:Fault xmlns:fault="http://www.medplus.com/hub/observations/fault"><faultcode>fault:com.medplus.serviceHub.common.exceptions.InvalidHl7Message</faultcode><faultstring>Invalid hl7 Message.</faultstring><detail><ErrorMessage errorKey="Invalid hl7 Message.">Invalid hl7 Message.</ErrorMessage><ExceptionClass>com.medplus.serviceHub.common.exceptions.InvalidHl7Message</ExceptionClass></detail></env:Fault></env:Body></env:Envelope>
    

    It’s shocking to me how different this is from their example and docs…. but whatever. Moving on. Since I used the IDs from their sample message it’s not surprising we got this kind of error.

    But even when I replace it with what I suspect is the data they want, I’m still getting that error. I think I need to do some nested document_id stuff.

    .h3 Update

  • MapQuest and AngularJS

    For my current project, I need to show MapQuest maps next to an address form. As the user enters an address, I need the map to attempt to show the address.

    Note: When testing, I often use console.log… so if you don’t have developer tools installed, some of these won’t work for you.

    Step One: Basic Gist

    Using this basic angularjs gist as a starting place, I added the basic code from MapQuest’s open data API. It works!

    Source
    Outcome

    Step Two: using AngularJS templates ( the single-page-app thing )

    In this case, you can see that the angular app is active… the word “World” replaces {{name}}. Importantly, though the content here is generated on the server side, not the client side. In my project, the map must be rendered by angularjs on the client side as part of a “single-page-app.” This has something to do with HTML5 and hashtag-urls. I’m still not totally comfortable with these concepts yet, but I have seen their coolness and am getting there.

    Steps to transition this from server-side to client-side rendering:

    • Move the content to a template file (map.html)
    • create a router that will recognize the current location and pick the right template to render
    • add <div ng-view/> or equivalent on the server side so Angular knows where to place its content.

    Here are the changes:

    Outcome

    Not surprisingly, the map disappears. Note this line of javascript:

    1
    
          MQA.EventUtil.observe(window, 'load', function() {
    

    so the map is drawn on window load. And angular is still loading so the #map div isn’t available at that time. You can tell that the map.html view is being rendered, though, because “Here is the map” acts as a canary.

    Step Three: get my map back.

    based on the referenced Stack Overflow pages, looks like $routeChangeSuccess might be a winner.

    Diff

    However, the $routeChangeSuccess event is called twice. The first time, it is unsuccessful, since there is no #map present. ( this is the redirect in the router. )

    Here’s one possible fix. It won’t work once we have multiple routes, though.

    Step Four: Add Address Form

    Step Five: Map Address

    Adding ng-model=“location.zipcode” to the input tags for the form, we can then reference that data from angularjs via $scope.location.zipcode. We shouldn’t bother mapping until we have at least a complete zipcode:

    For diff, see https://gist.github.com/mustmodify/5533979/revisions revisions 676bf92 and 49f3f96

    Step Six: Cartography

    This is actually version 3 of our app. Previous versions integrated with MapQuest with heavy-handed jQuery. Instead of reusing that code I’m investing time in getting a MapQuest situation working… hopefully that pays off. It’s a risk, but I think a useful one.

    AngularJS has Directives which, in theory, should allow me to abstract away the setup to a plugin.

    code: http://plnkr.co/edit/lEvDK8NrvR2K5Ephsy7e

    References

    Writing AngularJS Services

    AngularJS docs

    ngResource restful resource part.

    angular.extend may help me add methods to models.

    AngularJS Directives

    GoogleMaps AngularJS Directive

    Ways to display webpages that use angularjs and multiple files

    Plnkr.co — this is cool because you can (a) reference gists (a la http://plnkr.co/edit/gist:5533979?p=preview )… this is how the people on #angularjs ask you to present your questions so they don’t have to guess what you’re doing. However, you can’t embed code from a gist in your blog… you must create a “plunk” and use that. Also, doesn’t seem to be able to reference previous versions by commit number.

    bl.ocks.org allows you to render github gists based on a gist number, OR a gist number and a revision number. Uses github’s URL structure…. which I guess isn’t as much a feature as it gives me the impression that it’s a lightweight, and by implication hopefully well-crafted and stable, layer. Plnkr’s embeding doesn’t add anything to the page… just shows the content from index.html. bl.ocks.org includes title, your output, and then your code. So a bit less useful for iframes, but great for linking. For instance, try http://bl.ocks.org/mustmodify/5533979/1e4fac2390cadc16c4bf78ba4c83ba582ad1ec6b

    On Events

    SO: AngularJS: how to run additional code after rendering a template

    SO: AngularJS, how to watch for a route change

    SO: AngularJS: Handling route changes

    Events Not Documented

    On Mapping

    MapQuest Open SDK API Docs — mostly harmless

    Automating Reports from Misys Tiger

    My client has asked me to automate the monthly generation of reports from AllScripts/Misys Tiger.

    I have a list of reports that must be generated:

    • Daily Recap – for a given month, show one row for every day
      • Day of Month
      • Charges
      • Misc Charges
      • Charge Adjustments
      • Insurance Write-Offs
      • Net Charges = charges + misc_charges – charge adjustments – Insurance WriteOffs
      • Balance Transfer
      • Personal Receipts
      • Insurance Receipts
      • Total Receipts = Personal + Insurance
      • Receipt Adjustments
      • Ending AR
    • Doctor’s Financial – For a given year, show one row for every month of the current year. Then yearly totals. Then show the same table for last year.
      • Day of Month
      • Charges
      • Misc Charges
      • Charge Adjustments
      • Insurance Write-Offs
      • Net Charges = charges + misc_charges – charge adjustments – Insurance WriteOffs
      • Balance Transfer
      • Personal Receipts
      • Insurance Receipts
      • Total Receipts = Personal + Insurance
      • Receipt Adjustments
      • Ending AR
    • Procedure Analysis — totals for doctor/location and for doctor
      • Doctor
      • Location
      • Procedure ( Post Operative; Trigger Point I; ADX; Microscope, Hospital Visit )
      • Month To Date
        • Units
        • Dollars
      • Year To Date
        • Units
        • Dollars
    • Receipt Analysis —
      • Doctor
      • Category ( Personal Receipts, Insurance Receipts, etc ) and then TOTAL
      • MTD Total
      • YTD Total
    • Aging by Dr. Totals ( age by posting date )
      • Doctor ( and then grand total )
      • Pending Total
      • 0-30
      • 31-60
      • 61-90
      • 91-120
      • 121-150
      • > 150
      • Patient Balance
      • Total Balance
    • Aging by Dr. Detail — separate report for each doctor
      • Patient
        • Type/Cycle/Number
        • Name
        • CS
        • Pending Insurance
        • Aged Total
          • 0-30
          • 31-60
        • Patient Balance
        • Total Balance
    • Dept. Analysis
      • Doctor Name
      • Department ( Evaluation and Man.. 99201, 99202, … )
      • Procedure Name ( Office Visit, Second Op, Consult, Ear Molds, etc)
      • MTD
        • Units
        • Dollars
      • YTD
        • Units
        • Dollars

    The client remotely connects to the Tiger server, where she runs the reports. In October of 2011 she contacted Mysis asking whether these reports could be generated automatically. The response was that because it required sending parameters, she could not do that.

    To complicate matters further, some of these doctors need visibility on a group level.

    Options

    I’m considering a few options.

    • Interfacing with AllScripts/Misys Query++ to generate the reports by command-line or … other … api, though the idea that they would provide an API seems laughable +
    • interfacing with the main Tiger code. Definitely not my first choice.
    • creating a web service to automatically generate these reports directly from the database.

    Questions

    • What is Cognos and how does it fit in with Query?
    • Is there a CLI or API for Query?
    • Where are the report definitions?
    • Can I reproduce the production of these reports based on their report definition and mimicking their internal process?
    • If not, what data feeds these reports? Exceptions? What calculations are involved?

    ++ the worst-name-ever for a reporting tool.

    + Interoperability seems more like tooth pulling than the default. I think this has to do with the personality of winforms.

    Things I miss about Ruby when working in C#

    Preface: nil

    for those of you who don’t know, nil is like NULL, but way cooler. But if you don’t know, just pretend it means NULL, but if you ask whether NULL is true or false, it will say, “Oh, yeah, I’m false.” Yes, I know, if you haven’t heard of nil this makes you think I’m on crack. Roll with me.

    Returning a value or “Sorry, no.”

    I miss being able to return either the value OR nil, meaning, “no.” For instance, I have people picking a date from some dropdowns on a form. Then I need to (a) know whether it’s a valid date and (b) know what date it is. Ignoring, for a minute, that this problem wouldn’t exist in ruby, I could do this.

    Note: begin/rescue is ruby’s try/catch.

    1
    2
    3
    4
    5
    6
    7
    8
    
    def selected_date
      begin
        Date.new( year, month, day )
      rescue InvalidDateError 
      # I have never had to do this so I don't actually know *which* error it would be.**
        nil
      end
    end
    

    but in C#:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    
                    private Boolean specifiedDateValid()
                    {
                            int year = int.Parse( yearTextbox.Text );
                            int month = int.Parse( monthTextbox.Text );
                            int day = int.Parse( dayTextbox.Text );
                            
                            DateTime ignore = new DateTime();
                            string datestring = month.ToString() + "/" + day.ToString() + "/" + year.ToString();
                            
                            return( DateTime.TryParse( datestring, out ignore ) );                        
                    }
    
    
                    private DateTime specifiedDate()
                    {
                            int year = int.Parse( yearTextbox.Text );
                            int month = int.Parse( monthTextbox.Text );
                            int day = int.Parse( dayTextbox.Text );
                            
                            return(
                                    new DateTime( year, month, day ));
                    }
    

    another example. I’m creating a search method. Search by parameters A, B and C. Well, I decide maybe I will also want the option of searching by A and B. So I’ll just leave C blank. But that will make C# flip out… in Ruby, you can just say, “Hey, is that value blank? Or nil? If so, just ignore it”.

    Not having to deal with Tedium

    I just spent 20 minutes PARSING A DATE. Yes, I realize there are a ton of interesting things about that… but there are also a ton of interesting things elsewhere, and I should never have to deal with this unless my client is Doctor Who or whatever. Even though there are a lot of dates in Ruby, I have never had to deal with parsing dates. OK actually that’s not true… I did have to deal with it once. But that was because my users were entering dates like “3/310” … which was a typo and Ruby’s prebuilt tools were like, “Yeah, uh…. year? valid date? give me something I can work with here.”

    It isn’t fair to say I never have to deal with tedium… but honestly I would say it’s almost a daily thing with C# and about 90% better in Ruby.

    Enumerable Blocks and Tap and Blocks In General

    I feel like I’m sacrificing my effectiveness on the alter of infrastructure.

    So I recently wrote a Month model because I needed to model that. And then I wanted to get a list of months from month x to month y. For instance, right now I need all months from January of the previous year until December of the current year. But for the sake of completeness, I might want to go backwards… in fact, I anticipate needing that.

    In Ruby:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    
      class Month
        ... existing implementation
        def <=>(other_month)
          self.first_day <=> other_month.first_day
        end
      end
    
      # and then you could get a range by using Ruby's "range" syntax:
      first_month..last_month
    

    In C#:

    1
    
      # I'm not sure. I know you would have to implement some kind of interface... perhaps iComperable if memory serves? And then I guess you could do a for loop, but then you wouldn't know for sure whether you needed a < or > operator. 
    

    Things I just don’t like and/or don’t understand about C#

    In winforms, it seems like you can’t really get away from the codebehind… or whatever that’s called… the cs file behind a screen. Everything has to happen in that one file… from validation to parsing of values to … processing… a nightmare.

    Some things I miss about C# when working in Ruby

    - Date#ToString has some intuitive formats… on the other hand, Ruby’s way is more repeatable.
    - interfaces. Yes, that’s right. I miss interfaces. Ruby has duck typing… and that works for me. But sometimes I just wish I could … just … have interface guarantees.

    Some things I’m unsure whether to love or hate

    - “using” statements. On one hand, it’s great to know exactly what’s in there. On the other hand, some limited magic is part of Ruby, both for good and evil. But mostly for good. Depending on the dev.

    HL7 Issues Facing Developers

    Inconsistent use of HL7 Value Types

    Trying to increase quality control, I set told my import agent not to flag any observations where the value type wasn’t what my app would have selected as the value type. The results were surprising.

    Field Quest’s Value Type Quest’s Value LabCorp Value Type LabCorp Value Expected
    eGFR ST >60 ST >59 NM
    Vitamin D2 TX < 4 NM
    BUN/Creatinine Ratio TX NOTE NM

    So I guess the moral of the story is that Inequalities aren’t considered numeric by LabCorp or Quest. It might be ST or TX. And also ST and/or TX might be used for random other things like “Please see note.”

    Storing notes in the value

    Although I think it is overused, HL7 does provide a convenient note segment, NTE, that allows a lab to attach notes to an observation, patient, etc.

    I received the following observation (OBX) from Access Labs:

    value type: ST
    identifier: 127^CRP, Cardio^L
    value: 0.4 Low Risk of CVD
    units: mg/L+ (sic)

    Seriously. WHAT IS GOING ON HERE?

    1
    2
    3
    4
    
    OBX|59|ST|127^CRP, Cardio^L||0.8 Low Risk of CVD|mg/L+|                    |N||S|F||||||||||||ACCESS MEDICAL LABORATORIES|5151 CORPORATE WAY^^JUPITER^FL^334583101
    NTE|1|L|                         Low Risk of Cardiovascular Disease: CRP < 1   mg/L
    NTE|2|L|                         Medium Risk(<2-fold increase)     : CRP 1-3   mg/L
    NTE|3|L|                         High Risk(Approx.2-fold increase) : CRP  >3   mg/L
    

    koding.com advice on being a great consultant

    http://blog.koding.com/2012/08/freelance-developers-you-are-the-future-dont-mess-it-up/

    I thought this post was insightful, if repetitive. I recognized just about all of the issues discussed as things that have been problems for me at some point.

    • The need for professionalism
      • If you’re being paid, it isn’t a hobby. Act professionally.
      • Don’t disappear.
      • Communicate early and often.
      • Coworkers and clients should rarely know you work from home… ie avoid “hold on, neighbor is at the door.” or “my wife wants me to ______” ( still struggling to get the wife to accept this one. )
      • no one is watching you, so you have to be that much more vigilant about not skipping work, not being sloppy, etc.
    • Balance passion for work with a consultant’s distance
      • don’t fall in love with your task list.
      • Have great ideas, but don’t be upset if the client goes a different direction. It’s their money.
    • Consultants must balance home and work
      • stick to an 8-ish hour day whenever possible
      • Rest on weekends.
    • Get things done, but done well. Don’t seek perfection, but don’t commit junk. After reading “Good to Great”, I am fascinated with the idea that tension is a critical part of all good decisions.

    Misys Tiger via ODBC via C#

    The code below constitutes my various attempts to connect to Misys Tiger via ODBC using C#. Feel free to skip to the word “WiN” to skip the failures.

    Warning

    This code below is not inteded to be production-worthy. I just wanted to see data. Excuse the slop.

    Questions and Answers

    Q. How do I create a connection to the various ODBC connections available?

    A. see “#WIN” below.

    Q. How can I retrieve data from this connection? Some examples found online fail for this connection.

    A. see “#WIN” below.

    Q. How should I handle the numerous errors that can come from the ODBC connection?

    Q. My client’s installation of Tiger has many companies. Each company has its own ODBC connection. ( Company_0001, Company_0002, Company_0012, … ). Each time I esablish a connection to a new company, a dialog asks the user to authenticate. Can I make it so that the user only has to sign in once?

    A. The ODBC connection respected the username and password when embedded in the connection string:

    1
    
    DSN=Company_0001;Uid=username;Pwd=password
    

    ADO.NET with DataAdapter – #FAIL

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    
                    public class CompanyListFactory
                    {
                            public static DataSet Get()
                            {
                                    OdbcConnection _connection = new OdbcConnection("DSN=Company_Shared");
                                    _connection.Open();
    
                                    DataSet ds = new DataSet();
                                    OdbcDataAdapter query = new OdbcDataAdapter( "select * from root.COMPANY_APC", _connection );
                                    query.Fill( ds );
                                    
                        _connection.Close();                                
                                      return( ds );
                            }
                    }
    

    I get this error:

    1
    2
    3
    4
    
    System.Data.Odbc.OdbcException: ERROR [IM001] [Microsoft][ODBC Driver Manager] Driver does not support this function
       at System.Data.Odbc.OdbcDataReader.NextResult(Boolean disposing, Boolean allresults)
       at System.Data.Odbc.OdbcDataReader.NextResult()
       at System.Data.ProviderBase.DataReaderContainer.NextResult()
    

    I suspect this is a valid error because when I change the table name to COMPANY_XYZ I get:

    1
    
    [Transoft][TSODBC][usqlsd]Unknown Table 'root.COMPANY_XYZ'
    

    After reading about ODBC compliance, I suspect the OdbdDataAdapter is issuing commands that include pagination, etc., which perhaps the Transoft driver may not respect.

    ODBC Compliance

    In a previous, aborted attempt to connect, I got this error:

    1
    
    System.Data.Odbc.OdbcException: ERROR [01000] [Microsoft][ODBC Driver Manager] The driver doesn't support the version of ODBC behavior that the application requested (see SQLSetEnvAttr
    

    which led me to the understanding that there are levels of ODBC conformance. Microsoft is assuming they are using the “we are awesome” level of ODBC… but it seems more likely that they are using the “bare minimum necessary” level of ODBC. At this point what I want is to go back to the ADODB model… “I send you a select, you send me back raw data” model, rather than the “Let’s have handlers and all kinds of levels of abstraction” model.

    Update

    The Transoft U/SQL Help Document provides a relevant table starting around page 70. This document even provides C# example code on page 113 !

    Using COM component, just like the example: #FAIL

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    
                    public void UseComComponent()
                    {
                            ADODB.Connection conn = new ADODB.Connection();
                            conn.Open("DSN=Company_Shared");
                 String sql = "select 'A' from root.COMPANY_APC";
                
                            ADODB.Recordset rs = new ADODB.Recordset();
                            rs.CursorLocation = ADODB.CursorLocationEnum.adUseServer;
                            rs.Open(sql, conn, ADODB.CursorTypeEnum.adOpenForwardOnly, ADODB.LockTypeEnum.adLockReadOnly);
                            object zz = rs.GetRows();
    
                            while( rs.EOF != true )
                            {
                                    Console.Write(rs.DataMember);
                                    rs.MoveNext();                        
                            }
    
                        
                            conn.Close();
                    }
    

    An example I have been referencing was written in VB, so it used COM components intead of the CLR. I added a reference to the COM component “Microsoft ActiveX Data Objects 2.7 Library” (which the author of that post also used). That gave me ADODB. I was able to get pretty far, but couldn’t quite figure out how to get the data out of the reader:

    using ADO.NET’s DataReader — #WIN !!!

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    
    /* some of these aren't necessary, don't remember which. */
    using System
    using ADODB;
    using System.Data;
    using System.Drawing;
    using System.Windows.Forms;
    using System.Data;
    using System.Data.Odbc;
    
                    public void UseAdo()
                    {
    
                            OdbcConnection _connection = new OdbcConnection("DSN=Company_Shared");
                            _connection.Open();
                            OdbcCommand cmd = new OdbcCommand("select 'A' from root.COMPANY_APC", _connection);
                            OdbcDataReader reader = cmd.ExecuteReader();
                            Console.WriteLine("JW Was Here");
                            if(reader.HasRows)
                            {
                                    while(reader.Read())
                                    {
                                            string s = reader.GetString(0);
                                    }
                            }
                            
                _connection.Close();        
                    }
    

    Even though I had already failed to use ADO.NET, I tried again. This time I found an article that went into great detail about various ways to retrieve data.

    Various Errors Experienced, some solved.

    Another Computer, Another Error

    I built a spike and took it to another computer, running XP instead of Windows 7. I got this error:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    
    ERROR [01000] [Microsoft][ODBC Driver Manager] The driver doesn't support the ve
    rsion of ODBC behavior that the application requested (see SQLSetEnvAttr).   at
    System.Data.Odbc.OdbcConnection.HandleError(OdbcHandle hrHandle, RetCode retcode
    )
       at System.Data.Odbc.OdbcConnectionHandle..ctor(OdbcConnection connection, Odb
    cConnectionString constr, OdbcEnvironmentHandle environmentHandle)
       at System.Data.Odbc.OdbcConnectionOpen..ctor(OdbcConnection outerConnection,
    OdbcConnectionString connectionOptions)
       at System.Data.Odbc.OdbcConnectionFactory.CreateConnection(DbConnectionOption
    s options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection own
    ingObject)
    

    My initial reading of this is that it tried to open a connection and one of the commands issued failed(?). I opened Misys Query on this machine and re-ran it. Successful connection. I closed Misys Query and the program continued to work. My suspicion is that Misys Query is creating ODBC connections that aren’t there when the computer first starts. Needs further research.

    Can not use DataTable.Load( DataReader_instance )

    Interesting sidenote: When I put a breakpoint at DataTable.load( reader ), and then I step past it, I get nothing instead of this error.

    1
    2
    3
    4
    5
    6
    
    ERROR [IM001] [Microsoft][ODBC Driver Manager] Driver does not support this func
    tion   at System.Data.Odbc.OdbcDataReader.NextResult(Boolean disposing, Boolean
    allresults)
       at System.Data.Odbc.OdbcDataReader.NextResult()
       at System.Data.DataTable.Load(IDataReader reader, LoadOption loadOption, Fill
    ErrorEventHandler errorHandler)
    

    References

    AllScripts Tiger to MS SQL using GoDaddy Hosting
    ADO.NET code samples
    ADO.NET for ADO Programmers
    Retrieving Data Using the DataReader
    DSN Connection String Samples
    Transoft U/SQL Help Document

    Finding false negatives / false positives in unit tests

    I get a lot of confidence from seeing my tests fail, then pass. Recently I wondered whether I shouldn’t be codifying that. Just because I saw it fail initially doesn’t mean I’m not getting a false positive now.

    Caution: this is purely theoretical. It’s worth exploring on a blog, but the agile part of me realizes that I rarely have issues with false-positives in tests. So don’t start trying to codify potential false-negatives because you read it on a blog somewhere.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    
    it 'exposes color' do
      positive do
        Animal.new('frog').color.should == 'green'
      end
    
      negative do
        Animal.new('frog').color.should == 'pink'
      end
    end
    

    So the interesting thing is that the failure I often see is that I haven’t coded something yet. Subsequently, I can’t really codify a false positive / false negative situation.

    Nothing to see here. Move along!