Shelter

Tech, philosophy and random musings

by

Boundless?

Hehe, had to giggle a bit when reading this (in my never-ending quest for semantic mapping of software systems architecture) ;

“The Open Group is a vendor- and technology-neutral consortium, whose vision of Boundaryless Information Flow™ will enable access to integrated information within and between enterprises based on open standards and global interoperability.” (My emphasis)

To embrace “open” you would think that getting to the info would be, you know, just open, but no. There are free chapters available for you to see, just to make sure, you know, there’s an engine under the hood, all you need to do is to jump through a few hoops, register, and … ugh. So I bit this bullet, registered and I got the introductory guide to the TOGAF framework, but it reads as any other fluffy “use our framework, and all shall be well in the world” vendor selling you miracle cures out there. Disappointing, really. I could get the full thing if I apply for a 90-day personal license, but I don’t think I’ll bother as I react badly to fluff, and I definitly get an allergic reaction to having to register, revealing personal info and such, just to get to read their fluffy bits. What gives?

I think these people have completely misunderstood what “open” means, which is a shame considering that the content might be useful. But I say only might, as I would have no idea. Phft, open, my ass.

by

Missing ontological serinity in the world of software systems architecture

Updates: See bottom, but also this question on StackOverflow.

Ok, so let me say from the get go that I’m a little bit upset. Well, maybe angry and bewildered more than upset, but nevertheless not happy. And it all has to do with the dingbat way we architecture our various computer systems. So, yeah, quite generic and not really something we can do much about.

Let’s rehash. I’m a SOA junkie, an EDA pimp, and I hate by default the bullshit in any Enterpise camp that promotes their way of doing must be right. And by SOA, I don’t mean no ESB bullshit, I mean a hard-core focus on services for architectural means. I build ontologically driven systems, and care deeply about semantics where most others don’t give a monkey’s bottom.

Lately I’ve had to rehash my knowledge on plugin architectures (both implementation specific and theoretical), how to modularise complex pieces of software, and implement an event-driven platform on which to run my systems. So I’ve been snooping around, and there’s a ton of models and architectures to be found. But being found is not the same as finding what you’re after, especially as I have a few criteria to my search; I want to find something that’s generic, simple (but not simplistic), elegant (as in, does not suck) and extendable, an architecture that’s event-driven, modular and open. Nothing. I’ve found nothing. Of course they all claim to be amazingly fantastic and super and great, but looking under the hood, if allowed, reveals yet another staticly created shared library stack with some hooks for your software to use, using some misnomer like SOA or EDA or any of the hundreds of other Enterprise bullshit terms out there.

So, I set my goals lower in the hopes of finding anything of value, even went and asked real programmers what I thought was a simple question, making it specific enough to hopefully muster some replies. Nothing. It seems everybody’s got their own way to handle their own little piece of the universe, that people cling to their silos of comfort or something, afraid of what might happen if we all agreed on something. Even when you dig into large architectures, like my own Linux Kernel which I’m using to write this post, there’s tons of layers and shared libraries that’s hubbled together in a way that does the job, ok, but doesn’t make it, in my eyes, an easy job to do, elegant to extend or easy to change.

I guess I should clarify. I’m knee-deep in ontology work for software systems architecture, a field that’s almost chemically free of any active community, has a few scattered experiements that went no where (and I’m tempted to put ADL in that category, too), a few papers here and there that talks about it in very generic terms (either as abstracts to academic stroke sessions, or a white paper claiming to be the second coming of Jeebus!), but as to hard-core practitioners like me who want to inject a Topic Map with events of given types that matches certain ontological expressions and Topic Map fragments of certain types of architectual patterns, tough! You’re on your own, kid.

So, what am I after?

Well, many things, but I’ll try to be a bit clear here. I’ve cut down on my wants, to, in order to try to find others out there doing similar things. So. I’d like to see a simple event-driven software stack that scales ontologically, and isn’t bound to any technology, company or otherwise religious platform. This means that the stack with its names and values work just as well for a small plugin as it does for a larger system like an extra-OS or a cloud, works for potato-peelers as well as online booking agents, database connection pools and kernel space memory managers, but also can grow and shrink with need, in such a way that all other parts of it when they need to can find out what those changes are. This digs into creating an upper ontology for information science, of course, but more importantly it means I’d like to plug software into various parts of a stack, so that everything – and I mean everything! – is an event listener. I know some micro-kernels work in similar ways but highly statically bound, but regardless these ideas are way past the cradle stage by now and need to have a greater exploration in the real-world.

So when I download an open-source package of sorts and try to find out what its stack of operation looks like, why is this information so hard to find? Or compare the Java event model and the .Net model. Or OSes. It seems it’s very hard to agree on these things, but I doubt the state of things isn’t because they’ve tried and failed, but because they haven’t tried. It’s a big world and this is a big field, yet this has not been tried in any meaningful way.

Sure, the technologies promoted through OASIS, ECMA and W3C in themselves have various solutions and tries to bind stuff together in a coherent way as not to confuse us too much, but even within their own stacks of proposals and standards there are huge gaps, great leaps of faith, and generally no clear direction. Even W3C who pushes the semantic web movement hasn’t got anything to say on the matter. It’s starting to drive me bonkers.

Ok, I’m done. My steam has gone out, but I’m not feeling any better. Off to do my own thing, like the rest of them. 🙂

Update: Ok, it seems I’m not getting my message across. Let me create a simple (and wrong) example ;

  • SOA : Start
  • SOA : Configure
  • SOA : Map
  • ENV : Start
  • ENV : Configure
  • ENV : Map
  • APP : Start
  • APP : Configure
  • APP : Init
  • APP : Connect
  • APP : Perform
  • APP : Teardown
  • ENV : Teardown
  • SOA : Teardown

Here we got an application session events where SOA is, er, SOA, ENV the “environment” (whatever that should mean), APP is an application, and so forth. This list should be HUGE! Think of all the interesting events one could generate from you turn the computer on until someone gets Rickrolled on the other side of the planet! I want to map environments, systems and eco-systems, with labels. In some regards it’s an enumerated list of points that any computer system traverses on its path from being loaded into memory until it leaves it. And possibly then some.

I want to map the software system world! I want to know what people call their various points on the software stack, what they call their events, how they see them work together, how they forsee workflow interactions, how they define system integrity, thoughts on implementation, named entities, the works.

I can find heaps of this stuff, but none of it is globally agreed upon, it’s all tucked away in projects or companies, it’s their own version of how things should be and what happens. Even big players such as Sun / Java and Microsoft / .Net have very different event models and ontologies, and they are not compatible in any meaningful way. I would expect some parts of CORBA had done work in this area, but what I’ve seen is very transaction oriented where clients already know the ontology and uses CORBA to travel through rather than be defined by.

As an example of the closest I’ve found so far in the realm of mapping machine-parts (“machine” here is “software systems”) is the Open architecture computing environment which tries to define up the most important parts of software systems (although the final version was released in 2004 … these things can be considered to be final? Where’s the clouds?), but lacks the ontological and semantic definition, has no event or message structures or standards, nor does it have any notational value or end-points which, admittedly, I could spend the next couple of weeks doing, but let’s see what else is out there.

Making any sense?

Update: To be even more specific, trawling through IPC is really what I have been doing for the last few days, but getting to the core ontology of all of this is soooo painful. Surely someone out there have done something like this? I’ve even gone through POSIX trying to gleam what nuggets I could find, but the system level of that beast is just so low it’s not funny. Promising is the DBus architecture and event stack, but this again is very low-level, covers only a fraction of the software systems, and is littered with duplication of complexities.

Anything else I should hack at? Yes, I’ve gone through the most of the WS-* stack as well, digging into past knowledge I had hoped to never see again, but here as well as most other technologies out there they seem to be obsessed with being so flexible that they forget to be defining. So, we get a lot of scaffolding and frameworks that you can extend and define your stuff in, but no clear definitions of what the world looks like. Even an obvious contender like WS-Events and the less-know WS-Event from Hewlett-Packard have nothing more than a functional approach to defining and registering events but that’s it, leaving the defining to some semi-ontological layer.

But I’m still convinced lots of people have done this sort of work, especially in these Semantic Web haydays. Browsing through the thousands of OWL ontologies in Swoogle for ‘software architecture’ (which doesn’t really cut it, but is the closest term that yield results) leaves me just overloaded. Sure, the OpenGroup SOA ontology for example, does provide me with, eh, lots of interesting stuff, but again it’s a special domain (SOA, obviously) using a certain moniker (service orientation, which sucks when you want to define events across operational stacks).

Argh! Can you tell I’m going bonkers?

by

Ressurection : xSiteable Framework

I’ve just started in my new job (yes, more on that later, I know, I know) and was flipping through a lot of my old projects over the years, big and small, and I was looking for some old Information Architecture / prototyping tool / website generator application I made with some help from IA superstar Donna Spencer (nee Maurer) back when I lived in Canberra, Australia.

I found three generations of the xSiteable project. Generation 1 is the one a lot of people have found online and used, the XSLT framework for generating Topic Maps based websites. I meant to continue with generation 2, the xSiteable Publishing Framework (which runs the Topic Maps-based National Treasures website for the National Library of Australia) but never got around to polishing it enough for publication, and before I came to my senses I was way into developing generation 3, which I now call the xSiteable Framework (which sports a full REST stack, Topic Maps. And yes, I’m still too lazy to polish it enough for publication (which includes writing tons of documentation), at least as of now, but I showed this latest incarnation to a friend lately, and he said I had to write something about it. Well, specifically how my object model is set up, because it’s quite different from the normal way to deal with OO paradigms.

First of all, PHP is dynamic, and has some cool “magic” functions in the OO model which one can use for funky stuff. Instead of just extending the normal stuff with some extras I’ve gone and embraced it completely, and changed my programming paradigms and conventions at the same time. Let’s just jump in with some example code;

// Check (and fetch) all users with a given email
$usercheck = $this->database->users->_find ( ’email’, ‘lucky@bingo.com’ ) ;

Tables are are contextually defined in databases, so $this->database->users points directly to the ‘users’ table in the database. (Well, they’re not really table names, but for this example it works that way) The framework checks all levels of granularity, and will always return FALSE or the part-object of which you want, so for example ;

// Get the domain of a users email address
$domain = $this->database->users->ajohanne->email->__after ( ‘@’ ) ;

Again, it’s like a tree-structure of data, a stream of granularity to get in and out of the data. This does require you to know the schema (and change the code if you change the schema), but apart from that, in a stable environment, this really is helpfull (it’s also cached, so it’s really fast, too).

You might also have noticed … users->ajohanne->email …. Where did that “ajohanne” bit come from? Well, as things are designed, again the framework will try to find stuff that isn’t already found, so “ajohanne” it will automatically look up in designated fields. All objects that extend the framework have two very important fields, one being the integer primary identifier, the second one the qualified unique name (so not a normal name as such, but a most often a computer generated one that isn’t normally a number. Often systems will use things like a username, say, as a qualified name, and hence “ajohanne” was my username in one such system). Why do this?

Well, PHP is dynamic, so in my static example above, explicitly using ‘ajohanne’ as part of the query, isn’t the best way to go in more flexible systems, but just pop your found user in dynamically instead;

$domain = $this->database->users->$username->email->__after ( ‘@’ ) ;

Easy. And this applies to all parts of the tree, so this works as well ;

$domain = $this->database->$some_table->$some_id->$some_field->__after ( ‘@’ ) ;

No, from the two examples above we might see a different pattern, too. All data parts has unrestrained names, all query operations use an underscore, and all string operations uses two underlines. (__after is a shortcut for substr ($str, strpos ( $str, $pattern ) ), and I’ve got a heap of little helpers like this built in ) Through this I always know what the type of the object interface is, and with PHP magic functions these types are easy to pull down and react to. As some of my objects are extendable, I need to pass _* and __* functionality up and down the object tree.

Traditionally, we use getters and setters ;

$u = $obj->getUsername() ;
$obj->setUsername ( $u ) ;

I turn them all into properties, so ;

$u = $obj->username ;
$obj->username = $u ;

But they are still full internal functions to the object, and this is another magic function in PHP ;

class obj extends xs_SimpleObject {
function getUsername () {
…
}
function setUsername ( $value ) {
…
}
}

The framework isn’t just about object persistence. In fact, it is not about that. I hate ORMs in the sense that they still drag your OO applications back into the relational database stoneage with some sugar on top. In fact what I’ve done is to implement a TMRM model in a relational database layer, so it’s a generic meta model (Topic Maps) driving that backend and not tables, table names, lookup tables, and all that mess. In fact, crazy as it sounds, there’s only four tables in the whole darn thing. I’m relying on backend RDBM systems to be good at what they should be good at; clever indeces, and easier joins in a recursive environment (which, when all data is in the one table, it indeed is recursive), where systems use filters to derive joins instead of doing complex cross-operations (which takes lots of time and resources to pull off, and is the main bottleneck in pretty much any application ever created which has a database backend.

A long time ago I thought that the link between persistent URI’s for identity management in Topic Maps and the URI (and using links as application state) in REST were made for eachother, and I wanted to try it out. In fact, that fact alone was the very inspiration for me to do the 3rd generation of xSiteable, hacking out code that basically has one URI for every part of the Topic Map, for every part of the TM API, and for other parts of your application. Here’s some sample URIs ;

http://mysite.com/prospect/12
http://mysite.com/api/tm/topics/of_type:booking
http://mysite.com/admin/db/prospects

At each of these there are GET, PUT, POST and DELETE options, so when I create a new prospect, it’s a POST to http://mysite.com/prospect or a direct PUT to http://mysite.com/prospect/[new_id], for example.

All in all, this means I have many ways into the system and its data, none of them more correct than the other as they all resolve to topics in the topic map. This lowers the need for error checking greatly, and the application is more like a huge proxy for a Topic Map with a REST interface. It’s a cute and very effective way of doing it. I’m trying various scaling tests, and with the latest Topic Maps distribution protocols that I can use for distributing the map across any cluster, it’s looking really sexy (although I still have some work to do in this area, but the basics rock!).

Anyway, there’s a quick intro. I guess I should follow this up with some more coded details of examples. Yeah, maybe next week, as I need to get some other stuff done now, but I like the object model I’ve got in place, and it’s so easy to work with without losing the need for complex stuff. Take care.

by

Thoughts on PHP

Yes, yes, I admit the herasy; I like PHP. No, no, PHP has tons of worts with it, so no, it’s not better that alternative X, Y or Z for task Q, W or E. I hate comparing languages this way, feature by feature, syntax by semantics, and so on. I like to judge languages on two things;

Environment

I really like the CGI style for web resources. No, PHP ain’t CGI (unless you’re in a pain-self-inflicting mood) but most often a glorious Apache module which reuse all the goodness it can offer, but the model of a totally independent scripting engine which needs to mold its relationships, and then you throw it away when you’re done, makes for a clean, fast and very scalable framework. PHP basically compiles together mostly C modules, and use its simple syntax to glue stuff together. Yes, there’s a backlog of really shitty and badly written code out there, especially when people have no clue. And when the threshold is as low as it is for PHP, that’s inevitable, but I hardly hold that against the language itself. The environment in which that shitty code runs is really good. To me, the environment is the best part of PHP. Perl falls somewhat into this same category.

For those in the know, this model is very similar to how a RESTful system works, where the interpreter is a manifestation of a resource. In resource-oriented development this means gold, and is very important to me as my environment supports my RESTful way of designing systems.

Style

And since style is something any good developer can control themselves (unless their language is super-strict), this really is the main thing that I like about PHP; It gives me the freedom to make things work for me. I can choose the methodology, whether a function, class or inline is best suited, and since PHP always is evaluated in run-time, I can make my environment depend on real-time parameters rather than a pre-compiled utopia.

Zend Framework

I’ve been using the Zend Framework for the last year or so, and it’s a great framework as such, although the focus has been mostly to put OO wrappers around common PHP idioms and conventions. As such it works great to perhaps consolidate the features, and perhaps give PHP 6 a future direction. This is the best part of the Zend Framework.

The bad thing about the Zend Framework is that it imposes its own style, and somewhat alters the environment. Ouch, on both the things that make PHP my choice tool. I’ve struggled quite a bit in trying to reuse certain parts of the framework, extend others, and generally use some bits without using the whole shebang. There’s not a lot of dependencies between the components, but just enough to make it tricky to do serious stuff (for example writing an alternative “threads-like” HTTP adapter to HTTP Request).

Now, people talk about the Zend Frameworks impact on the future of the PHP language. Yes, one can always hope that this is the case, but I think people are a little too preoccupied with the OO capabilities and forgetting perhaps what makes PHP really popular with those who choose to continue to use it past their first few applications. Do not fall into the trap thinking OO is somehow better than other ways.

As much as people think MVC is the best thing, I really don’t care about that. MVC works great for some things, not for all (for example, I have a REST framework that use a completely different model, a more resource oriented model), so to impose MVC as the modus operandi is not good, and indeed something that makes reuse of other ZF modules a bit trickier. Instead of a MVC focus, there should be a strong highlighting of a uniform interface to the environment itself. It’s the environment that’s cool with PHP, so let’s make interfacing to that better. There’s some work on wrappers and readers for various aspects of the HTTP protocol, but only the most basic stuff is in there and needs serious work. With PHP I could do serious HTTP applications; with Zend Framework I’m limited, and need to hack and extend. Let’s get HTTP savvy, not MVC drones.

Availability and diversity

PHP is everywhere. Period. I can use pretty much any ISP or in-house hosting to host most of what I need. And there are so many different open-source projects about that uses PHP (WikiPedia, WordPress, PHPBB, PHPMyAdmin, Drupal, Joomla, Flickr … the list goes on) meaning both hosting and the amount of high-quality tools are abundant. The LAMP stack is pretty much supported on every hardware and software platform. Heck, I can even run it on the JVM.

I have written many tools over the years, some good, some bad, but I’m always happy to find out that I can copy my old files into any new environment and they will pretty much always run straight away, no tweaking. I can’t tell you how important this is!

The shitty stuff

No system is perfect, to some specific definition of what “perfect” means. With PHP, I’ve learned to live with its odd bits, such as funny booleans and comparators, diverse and non-uniform APIs, sloppy exception handling, no shared memory (which may or may not be a good thing, but certainly a big part of its design), and a few syntactic snuffs (Why not introduce “$$” as a shortcut for “$this->”, for example? Tidbits).

Some say that PHP code is crap, usually said when encountering – indeed! – shitty code. But shitty code is everywhere. Even in the most tied-down, static and controlled environments you still get shit. Some say less shit. I say shit with a different color, not different odor. I’ve been programming for almost 30 years soon, and let me tell you, I’ve encountered shitty code in every single environment and language I’ve ever tried, and I’m one of those who tries every language out there. There is no language who can hold a stick to the wonderful imaginitive mind of man, able to lay bricks where diamonds should have shone.

Some say the PHP language itself is immature, or unstructured, or some other parameter that their favourite language holds. No, sorry, but there’s not that much which separates all our different languages (except BrainFuck … that one is seriously different. :), and indeed a lot of the language wars are really more about their API designs than the languages themselves. For example, the syntax of Java is tolerable, but a lot of its APIs are not. The syntax of Python is ok, but some of its APIs are great. Ruby has good syntax, but to me some confusing APIs.

It’s really mostly about APIs, and how we create models to solve our problems. It’s all about models, as APIs in themselves are models. The API is not the language.

Round off

I should here of course mention the link between models and Topic Maps, as the latter is a way to define, work with and exchange / share the former. Couple this model openness with resource-orientation, and I think it makes for a very interesting environement. But as this whole shebang is part of the new framework I’m working on, expect more blogging on this in the very near future. Have a wonderful week.

by

SOA; ROA; WOA; REST; SOAP : A point about what we’re addressing

Hang on to your buzzword-safe underwear! There’s a lot of buzz these days in the IT architecture world about whether to pursue SOA, ROA, WOA, do it RESTfully or SOAPy, what bus to drive, and what governance you can throw on top to make it all work.

I am a RESTafarian myself, and everything I do I do in a Resource Oriented way, so you can say I prefer the Resource Oriented Architecture (ROA) which really is a synonym to Web Oriented Architecture (WOA). The SOA way is far less technically defined, as SOA can be created with almost any technology, but there are some traditions and current marketplace geist with the term that associate it with larger stacks that use the SOAP and the various WebServices standards as a mean to “put it all together.”

Right, what’s the fuzz all about? Why are some throwing themselves at the resource oriented way while others stay with the API oriented way?

Did I say API oriented? My golly, I think I did, and this needs some explanation, and I’m going to quote from an upcoming article I’m writing which include a section on what I call the reversal of the address-function space ;

Let’s talk about design of your systems for a minute, as the answer is that all you know about governance, from IT governance to SOA governance, needs to have the same type of reversal of how you think about the services themselves and their individual evolution throughout your SOA space. You’re not governing API design, identity pollination, semantics across APIs and functional silos that addresses “access to things”: you’re governing how to express the “things” themselves. Thinking that services starts and ends at given points in your ROA is severly limiting your options, especially in terms of expressiveness.

This reversal is well known to functional (not to be confused with imperative) and declarative developers, where the focus is on the operational expression of results as opposed to executing a series of commands to get there. Within the RESTful community we often talk about how we use nouns instead of verbs, where we address “things” and relationships instead of “actions” and functions, and we need to be aware of both these angles into how we work with our systems.

If you thought these notions were small and somewhat insignificant, I’d beg to differ. Once you travel down the path of functional operations it’s hard to look back. Indeed, the combination of the reversal of the addressable function space and resource orientation is the very thing that makes the unified interface work so well, and that very interface is extremely important in terms of performance and scaling of systems, but there’s one more thing that needs to be addressed, pointed out and shouted, which is the semantic goodness of pointing to something and say “I want it.”

This is the way we humans foremost operate; first we point, then we interact. We don’t say “You know, cars? Well, you know all blue cars? Of those, there’s some that goes quite fast, and have really cool spoilers? Some of these are of the brand BMW. And of them, you know those that have the letters X, Y and Z on them, and really looks a lot like the car in front of us right now? You see that one? I’d like one of them.” No, we point at the car in front of us, and say “I’d like that one, please.” We don’t start with the generics of our conversations and traverse down until we find the specific thing we’re after, so why should we do this when dealing with computer systems as well?

APIs – or, I should say, the culture that emerge from it – focus on processes, but quite often we’re not interested in processes but results. Why are we defining processes instead?

SOA and SOAP / WS-* are highly process-oriented ways of thinking (“first do this, then that, then this, and finally that“), grown from the RPC (remote procedural call) philosophy of distributed computer systems. It was the natural way to start, but it doesn’t have to be the thing we should keep on doing. When we move into a resource-oriented way of thinking, we’re not defining a process as such, but a result. In fact, we leave the process-part to the backend systems, perhaps as a constraint of loose coupling, and this is an extremely subtle but important point.

I’m not even sure it can be explained as much as experienced; when you shift your thinking from processes to results (and, this is why XSLT programming is so hard for many people; it’s a paradigm shift in thinking) there’s a lot of boilerplate code that magically disappears. Declaring where you want to be instead of defining how to get there is subtle, oh so subtle. I can’t say anything more than just try it yourself. Try something new. I dare you.