Last 2 days I’ve visited the 5th international SOA, Cloud & Service Technology Symposium in London. It’s a yearly event that features experts and authors from all around the globe. I just landed back in the Netherlands and since I can’t claim a jetlag I forced myself to make this blogpost before the next event is already starting (yeah I know, my life sucks).
Big disappointment for this years conference was the fact that well known SOA & Cloud author Thomas Erl could not make it due to family illness. It was my 1st symposium so I was really hoping to finally meet the big man himself. The days where packed with sessions in multiple tracks so it was impossible to see al interesting ones. Some sessions where even so popular that you couldn’t enter the room properly (example: Your Security Guy Knows Nothing by Paco Hope, but what would you expect which such an epic title ?)
To much to tell, but a small overview I made below of the 3 sessions I found the most interesting during this event. I think the presentations will be made available very soon as well.
Rob(ert) has 20 years’ experience designing and implementing IT solutions and this was noticable due to his great amount of experience. He is also author of the Service Design Patterns book (by now on my wishlist) as he admitted himself was a shameless plug to sell more books ;-)
Anyway. Robert session explained the attemps to design the perfect API, one that would never change. However this will never happen since both clients and servers evolve in different rates. So in the end, developers must always accommodate new requirements and coordinate deployments to prevent breaking stuff. This is not freedom he mentions.
The common cause for breaking changes:
- data types (string becomes integer)
- optional vs required
- hierarchial relationship
- sequencing (xml), of which Robert is not a big fan.
If you look at the bulletpoints only 1 conclusion: A lot of data modeling issues.
Robert takes 1 API (RPC) and explains coupling by an example (SOAP/WSDL). Client used procedure name (operation) and arguments (xsd) to contact server. This is “crazy high coupling”. When change needed: change service, create new wsdl, generate new client proxy, etc. As Robert states: fun fun fun! ;-)
So 2 options there:
Backward compatibility: this is most often used to serve older service consumers with the older version of the message. Becomes a big problem with lots of consumers.
Forward compatibility: service is able to process newer messages they don’t really understand. Clients that have moved ahead of the service. Typically occurs when message defined by external groups.. Problem with validating / transformation.
Open standards now-a-days prevent vendor locking, but you will always have coupling. This you can not eliminate it. So while managing to prevent breaking changes, your talking governance. But as we all know, this is easier said then done.
As Robert states we need a tolerant reader:
- lax or no schema validation over data binding
- favor surgical data extraction: xpath, jquery selectors
- enable unknown content to be forwarded
- be conservative with exceptions
And relaxed message structure:
- favor optional items over required items
- avoid strict sequences
My personal remark: I understand that this could be helpful with read-services, but for write services you would need more requirements regarding valid data being send to your backend systems. So I would assume write services should have less relaxed message structures then read services could use.
So Robert summarizes, the problem of Web APIs
- may be used by multiple clients, so impact high (1-n) when changes needed
- documentation becomes dated (JvZ: as seems to be the case in general)
- architect need to think ahead about client future needs
Consumer Driven Contracts (patterns)
- service consumer developer writes integration tests that express their expectations
- service owner incorporates these tests into the service test suite
- tests definition is hard, not only happy path
Belgium Integration Architect Ignaz Wanders talks us through his experience with service versioning in a SOA landscape. An issue I often ran into myself as well, so I was curious what other peoples experience and best practices are in these. Especially since Ignaz mentioned to me during high-tea he often made use of the Oracle Service Bus in his projects.
The general version principles he summarizes:
- clients should not be forced immediately, but gradual, retire gracefully
- support multiple service versions
An overview is presented about existing design patterns that reference to versioning which Ignaz recognized mentioned by Thomas Erl in his book “SOA Design Patterns“
- canonical versioning (standardized versioning, not how done)
- concurrent contracts, 1 service, multiple contracts. Targeted different clients, different usage
- compatible change: avoid breaking changes
- version identification: version number usages (x.y.z)
Identical to Robert sessions, also Ignaz mentions 2 service compatibility strategies:
- todays version, comp tomorrow
- complex topic, so nobody uses it
- xml schemas supports this, but it’s unreadable and so complex
- new version available, consumers with older version
- optional field added for instance, then schema validation is ok but service contract not
Ignaz also explains the 2 options for message validation (compatibility contracts) in a service landscape: design time vs run time .
Design Time (schemas: wsdl/xsd):
- version in tech service contract (namespaces)
- single governance
- new version, requires new client build
Run Time (schematron):
- version in non-technical service contract
- must be governed
- new version not necessarily requires a new client build or deployment
- a bit more flexible then design, but harder to maintain
Ignaz then continuous with the real-world common scenario:
- endpoint of services contains version number
- grammar based service contracts (wsdl/xsd) are used
- version numbers in xml namespaces, which is terrible (JvZ: i fully agree)
- Only a single service version in operation at any time
So to minimize the big bang to mediation
- design by contract, contract first instead of contract last (JvZ: no db schemas or backend system element naming in the service consumer messages, sadly something that you still often see is being used)
- don’t hardcode versions in your service landscape
- use rule-based (xpath/schematron) instead of WSDL/XSD (full message) (JvZ: Ignaz has the same opinion as Robert, I’m very stubborn so I’m still in doubt if I would agree on them both)
At the end of his presentation he pleads for a WS-Versioning standard for SOAP webservices. This would allow version mediators to be build into a service bus and frameworks. An example is that ?versions could be added to w3c standards as is now the ?wsdl standard
I know Guido as co-author (and main driving force) of our Oracle Service Bus Developers Cookbook project. As I met Guido earlier that conference I mentioned I probably wouldn’t be attending his BigData/NoSQL session because my knowledge of the subject was very minimal. However Guido told me that his session actually could be seen as a good introduction and overview of NoSQL world. So I was convinced, and mainly because I fully agree to what he mentioned as well in the start of his presentation: “Architects should look around for trends that seem to occur and which trends are becoming more and more interesting for them”. In heart, I’m still and always will be a technology nerd so these kind of sessions are very valuable.
Guido starts with an overview of the history of databases during the 60′s until now. From file based, hierarchical databases through SQL query relational databases and NoSQL nowadays.
He explains while relational databases we all know are great, but makes an overview of the different problems:
problem: complex objects graphs
- object/relational impedance mismatch
- map screen/object to relational schema (2 tables). We have hibernate
- performance issues: many rows, many joins, eager vs lazy loading
problem: schema evolution:
- adding attribute to object, have to add columns to table
problem: semi-structured data
- name value table, poor performance / lack of constraints
- serialize as a blob, no query
So then we arrive at the subject NoSQL (Not Only SQL) and its characteristics:
- they don’t use the relational data model and thus don’t use sql
- tend to be designed to run on a cluster
- tend to be open source
- schema less
- uses existing frameworks as Lucene indexes, etc.
Guido shows a picture with a central database (1 big relational with multiple apps) vs application databases (every app, it’s own db) and the positive aspects of multiple application databases:
- atomic, easier to maintain and evolve database
- ready for the cloud (easier to move around)
An overview of NoSQL Database types is shown (key/value, column, document, graph) and examples of the available products out there.
Guido closes his presentation with a picture showing 3 different data access architectures:
- consumer / SOA middletier / database: example we see a lot in SOA landscapes where adapters/services connect to the database and present data to the service consumers. Positive aspect is the control over the data
- html5 webpage example / REST with JSON / object database: he warns about going back 20 years. While it is fast development it’s also very strict and hard coupling. Governance is almost none, so in the end you will regret this.
- consumer (html5) / SOA middletier / NoSQL database: the way to go, usage of middletier to maintain control over your data and prevent hard coupling between consumers and data.
2 very interesting days where I had to conclude that not all sessions were evenly interesting for me (some I found to high level and/or to abstract). But enough interesting sessions with experience from the front line trenches. Another pleasant surprise was when I received a preliminary manuscript review edition of ‘Cloud Computing: Concepts and Technology‘ by Thomas Erl. More books to read, the pile is only getting bigger.