nodalpoint.org

There has been an ongoing debate in the semantic web community, for many years now, about whether identifiers for resources should be URLs or URNs. This debate has recently flared up on the HCLS mailing list in the form of URLs+1 LSIDs -1, and since some of you not in the know maybe wondering what all the fuss is about, in this post I will try to explain why identifiers must be URLs and why the semantic web is broken by design.

As an example, take the Uniprot record for SERPINC1:

http://www.expasy.org/uniprot/P01008

From the browser's perspective the only significant portion of this string is everything before the first colon: http. HTTP is of course the protocol the browser will use to fetch the html text file, or representation of the thing being identified, from the server. It can only do this once it knows the location of the server, so taking the URL and then finding the physical address of the web server is actually step one. DNS is the protocol used for this, in short, a series of name servers arranged in a hierarchy are used to turn the name into a location, in this case the address is 192.33.215.47. The browser then sends an HTTP GET request for the page /uniprot/P01008. And what you seen in your browser is a webpage detailing information about the protein SERPINC1. This should not be news to anyone.

Shifting the example slightly, imagine you are a semantic web agent, and not a human being. The above URL is meaningless, or opaque as the W3C specifications describe it. Suppose further that your agent capabilities include understand HTTP and DNS. No problem, resolve the name, fetch the resulting document and then figure out what you have just fetched. As an agent you do not know what the document is describing, you can't read like a human [1]. Text mining to the rescue you say, but unfortunately we are dealing with the semantic web. People who use the semantic web would very much like to use URLs to identify things other than web documents, for example biological entities like the actual physical protein SERPINC1 and not an HTML page describing data about SERPINC1. For example

http://www.example.com/protein/SEPRINC1 is a Protein

Here the URL is identifying the protein SERPINC1, not a webpage about the protein. So as a semantic web agent what do I do with this identifier ? Superficially it is no different to the previous one, but if I try and resolve it what would I expect to get back ? In theory the W3C's family of specifications describing the function of the the semantic web should outline a clear solution to this problem. But they don't. They remain vague on this point, and as a consequence people are just making it up as they go along.

One way to avoid the problem of trying to resolve a URL that identifies a concept rather than a webpage would be to ask for agent (or machine) readable metadata about the resource the URL identifies, before trying to resolve it. Is it a webpage ? is it an image ? or is it a concept that has no physical representation on the web. Logically you would also expect the W3C standards to say something about this, but again they don't. These two problems are the fundamental deal breakers for the semantic web. Unless these are solved (specified) clearly, it will never take off, period.

Besides these problems, people have other issues with URLs as identifiers. First among these is that people consider URLs to be locations, not identifiers and second they're just not permanent enough. So since we don't want to just locate concepts on the semantic web we want to name them, and we certainly don't want our names to change, people have gone to the trouble of inventing another identifier system: URNs or uniform resource names. It is at this point we bring LSIDs into the picture, because LSIDs are a URN naming scheme, coupled with a standard way for agents to ask the question: what does a LSID identify, a concept or some representation on the web ?

Thus, LSID solves all of the problems mentioned above, it allows you to use a name to identify a concept or a document on the web. You can use the name with public LSID resolvers to find the location of the document that the LSID identifies or if it is a concept you can retrieve machine readable metadata describing the concept. Furthermore, the name and the location are decoupled meaning that the names are location independent. And finally if you use the LSID specification the identifiers you mint and the documents/concepts they describe must be permanent and unchaining. The fatal flaw in this carefully laid plan is the need for wide spread adoption of LSIDs. This has not occurred in the many years since they have existed, and it is not for want of trying: they were backed by IBM :)

So if it is a reasonable technical solution to the perceived failings of the semantic web, why weren't they adopted ? Because apart from the two deal breakers I mentioned before, the rest are just what I said: perceived failures, not real ones. For example take names vs. locations, remembering the L in URL stands for locator, not location. URLs have the helpful property that they are names that also have a widely adopted infrastructure (DNS) in place that enables agents to resolve them to locations and fetch (HTTP) their representations. In other words a URL helps you locate the thing that it is naming, it doesn't mean that it is the location. Permanence is likewise only a perceived failing, no amount mandating permanence in a specification will make it so. Put simply it is a human thing. Organizations must be very stable and consistent in how they mint identifiers, usage of a specification cannot make an organization stable, only responsible people working in that organization can.

So what is the way forward ? The W3C must fix the first two problems, mainly by specifying a way to enable agents to get metadata about a URL identifier, before it tries to resolve it. They can do this by making metadata a first class citizen of the HTTP web. In other words HTTP must have an extra verb to get metadata about a URL an not just get the representation. Such a method, MGET is elegantly specified in URIQA by Patrick Stickler. The URIQA proposal has been around for quite some time now and like LSIDs has not had wide spread adoption. The argument against it is that millions of webservers around the world would need to be modified in-order to support this new method, making the whole thing a non-starter. This is why the W3C must get behind a solution, because at the end of the day the HCLS mailing list should be about discussing ways to use semantic web technology to improve the data integration situation, not arguing about the fundamental architecture of the semantic web itself. My opinion matters little, since I don't participate in any of these forums. Nonetheless, I conclude that the semantic web is broken, because the W3C has not made metadata a core part of the web and this is why I have lost interest in the semantic web.

[1] It could be an image too.
[2] There has recently been a resolution on this issue by the W3C, but the solution is still contentions.