any14.com | larsreinertsen

What if there was a way for access to data could be authorized everywhere. If you were authorized to access a piece of data you could get access to it wherever it happened to be located.

This is not the way things work at the moment for sure, but if it could be made to work in a convenient way, what should it be like ?

When the first web browser arose some 20+ years ago. Static html pages and other media and document files where available by calling a URL over the HTTP protocol. Security was added – at first pretty coarse grained: If you were logged in you could access pretty much anything. It got better.
Comprehensive tools became available to centrally manage all web access to any document, with the finest granularity. The writing of the rules of who should access to do what, when and where, could be delegated out to those who actually were in a position to know.
But crucially, these tools could only manage access to stuff directly under their control. Often operating in a reverse-proxy mode intercepting HTTP traffic. APIs were available through which other applications could tap into them to take advantage of access control rules contained in them, to do their own authorization. In this way the data under the control of a unified set of access control rules could be made corporate wide. Access to all of a data in a corporation being governed by rules maintained in one place. Everyone would play together in the same data security pool.
In practice this never happened. (re-)Writing applications to take advantage of the API of the chosen security software platform , was too expensive. Other tools emerged to export the security rules from one software platform to another, leaving them to do their own enforcement through their own rule infrastructures. This didn’t work very well because it was too complicated. Rules are fundamentally about meaning, and meaning doesn’t translate easily. Never the less this was an attempt to federate authorization.

Data protected by the same access control rule infrastructure is part of the same pool. A database is a single pool. It has its own internal security governing access to individual pieces of data contained in it, but has no reach outside. The database maintains it’s own list of who gets to access which column in what tables.
A server has it’s own internal arrangement for governing access to the data in its own file systems. It may also have access remote file systems. Some remote file systems would be on other servers, which would govern access (NFS, FTP, Samba etc.) and would therefore not be part of the server’s own pool.

If authorizations could be federated between pools all data would exist in one big virtual pool.
A virtual pool made of multiple physical pools; individual databases, file servers etc. At present this is difficult as there may be user federation between some data pools, but each pool has it’s one authorization, it’s own way to enforce access rules. The rules in one are not known, or directly enforceable in another. There is no federated authorization.

Lets further suppose that any piece of data in this virtual data pool, data ocean really, is accessible over TCP with a URI. The URI may have various formats depending on what type of physical pool is being addressed.
For example, this would be the syntax of an URI accessing a directory (LDAP) store

ldap[s]://hostname:port/base_dn?attributes?scope?filter

And this to access a individual file, using HTTP(S)

https://host:port/path

Access to one of the secure web reverse-proxies mention above, would look like this too.

The would be many others. Note that the username and password does not appear. There would not be any prompting for this information either.
Access control would be through PAML tokens, passed in the headers. A SSL handshake would take place to establish the requesting entity’s authorization for the tokens presented.
All physical pools are defined by the entity that control access to it, and all of these entities, be they LDAP server and file/web server in the URI examples above must be equipped to handle PAML tokens to verify the authorization for the request. Through the acceptance of these PAML token the pools together form a virtual data ocean. Any application can call on data anywhere else and present PAML tokens for authorization.

This leave quite a bit of scope for application architecture. The use of a PAML token require access to the private key of the user to which the PAML token was issued. Which means that if a user is engaged in a transaction with an application and this application needs access to data kept somewhere else on behalf of the user, the application can only present its own PAML token, not forward those it has received from the users. The user must at a minimum contact this other data store directly and engage in a SSL handshake. This way the user’s ownership of the public key is established for the benefit of the data store. The application can then pass the PAML token received from the user on to the data store and the store would now know that the PAML tokens are OK to use; or the user could make the data retrieval directly and pass the data to the application that needs it. Sort of like a data federation.

Note that PAML token are tied to data, not any particular host environment. Among other things this means that the requesting client may send the server a considerable number of tokens in order to establish authorization for all required data. The server will grant the union of all these tokens.

Demonstrating how to have secure email through the any14.com service.
Using text files and command line tools. Not for the novice users perhaps, simply a demonstration of capabilities.

To start with you download the any14.com code base (SmartServer.war.zip)
Both the sender and the recipient need an addressbook xml file. In here is the information on the other party: name, public key and internet alias. The internet alias should be unique. It doesn’t have to be, for reasons that will be clear further down, but it saves bother if it is.
Both parties also have a properties xml file containing their own information as well as their private key.

The sample file for addressbook for the mail.

The war file is unziped into its own directory. Open a command line prompt, DOS or bash as you prefer, though the sample commands below will be for DOS.

This example is hardly user friendly. It is not meant to be. But put a GUI wrapper around the commands and it is as user friendly as you like. The point is to demonstrate the possibility of sending secure messages over plain HTTP through a generic service like any14.com. The bit about generic is a key point. It is easy enough for the powers that be to block access to more specialized services not used by many people. But the more users a service( i.e. a website) has the less likely is it to be blocked: it has too many necessary functions. More over, and perhaps more significant: using an obscure service might attract attention in certain quarters. Attention that a user might not want. Using a generic service should help the user avoid that attention. Now, of course a generic service mostly won’t offer you what you need – which was why you used the obscure one in the first place. This is where any14.com comes it. It offers the service you program. Any14.com is an on-demand service generator.
In this example we are using the any14.com API locally, having downloaded it from them. And are just passing the messages through their server. We could also have called the API on their server. But the point was also to remove the need of having to trust a remote server. It could be monitored for all we know. In this example the messages are encrypted before they are sent and can not be decrypted by anyone other than the intended recipient. Which is we why are not much bothered about by who might read our messages while they pass through the network and remote systems, and even use plain HTTP.

The SmartServer.war.zip contains sample xml files for a demonstration. There are also two txt files containing the DOS command to be used on Windows. If you prefer bash or another shell on Linux, the modification would be modest.

Both the sender and the recipient have two static xml files: one containing address book information, the other a properties file containing the private information of the party in question such as the user’s private key.

Sender address book, SenderAddressBook.xml

<addressbook>
 <contact name="">
  <name>Recipient Lastname001</name>
  <alias>recipient.lastname001@domain001</alias>
  <internetalias>reallyobscureuseraliasforinternetuse008621</internetalias>
  <publickey>-----BEGIN PUBLIC KEY-----\n
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC+8G1x7itnnHGlN0kodreKF7cf\n
e8NWiHk8j/tDD49y4m+jsAbpUh0THj2OWTtQhayswUUG7TW4y4qvONdv3zfIOgz7\n
T5aeBHQCXb8Kk54ffFJC+NYiBwwCR9FhxJzy2AXfJweQFXzYJMh+iz7WLebfrmWi\n
rsHJ4ZPtP4SEW4QRWQIDAQAB\n
-----END PUBLIC KEY-----</publickey>
  </contact>
</addressbook>

Sender properties file, sender.properties.xml

<properties>
  <addressbookfile>SenderAddressBook.xml</addressbookfile>
  <name>Sender Lastname002</name>
  <internetalias>reallyobscureuseraliasforinternetuse005234</internetalias>
  <publickey encoding="none">-----BEGIN PUBLIC KEY-----\n
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC+8G1x7itnnHGlN0kodreKF7cf\n
e8NWiHk8j/tDD49y4m+jsAbpUh0THj2OWTtQhayswUUG7TW4y4qvONdv3zfIOgz7\n
T5aeBHQCXb8Kk54ffFJC+NYiBwwCR9FhxJzy2AXfJweQFXzYJMh+iz7WLebfrmWi\n
rsHJ4ZPtP4SEW4QRWQIDAQAB\n
-----END PUBLIC KEY-----
</publickey>
  <privatekey encoding="none">-----BEGIN RSA PRIVATE KEY-----\n
MIICXQIBAAKBgQC+8G1x7itnnHGlN0kodreKF7cfe8NWiHk8j/tDD49y4m+jsAbp\n
Uh0THj2OWTtQhayswUUG7TW4y4qvONdv3zfIOgz7T5aeBHQCXb8Kk54ffFJC+NYi\n
BwwCR9FhxJzy2AXfJweQFXzYJMh+iz7WLebfrmWirsHJ4ZPtP4SEW4QRWQIDAQAB\n
AoGAQlzaDTEUUOyk6yFuAexRGmPnt7+SQu8oBA6e683qs6lT1/aEUG/DvzNRfbox\n
zWjIfTqnPSMQ+utzhjchWbahchIt5QTk2qWIP3zPPaK0eTyvKGrlb65VKkwKfOLx\n
zBWu0wvOzdBnHxx5MlRAYCUa6jlBgIiDsis+GPYtRL7NO8kCQQDoAJpV3aEp/Sdg\n
KkXfdGq47uV9/6dOYcOSYOFU0DwF2M2DXTzFWz/3aUetwMGAzxXkgTKxHvH67FG9\n
evY1hUInAkEA0rB52X2mwkpiAMPJeDxdBO3Ws8DZZotDcCGpWk2c6O+VdvVn2ByQ\n
m1R7vlSPTG0k461vxAua1zWevd7nir3AfwJBAOBKzOqchNXyP6IR3FFm3FKD62x/\n
v8DgdVHLxazYUF8atbz9Y+YtDM9lgf2yL6EXE8RBP7/K+ggrde7otKUTetsCQCcd\n
WlHVcQsrEgzATHwFAOP0X+LZT8Mo4alEppM8S4A3UpFWOCQ4EWS1B5ArVLtEnA+k\n
2FYWLXX8326ynlv0pHMCQQDJwA/yBmYGcJA7qyF1M+VSXDNTg8vwY43kU9LpxfgR\n
sdKpCT7fnyMqj+O84a2gi/X78IBNNR7ZW1t5FIhcRbMn\n
-----END RSA PRIVATE KEY-----</privatekey>

  <localmessagestore>
    <encryption algo="AES" >
      <key encoding="UTF-8">TheBestSecretKey</key>
      <seed>AAAAAAAAAAAAAAAA</seed>
    </encryption>
  </localmessagestore>

</properties>

Recipient address book, RecipientAddressBook.xml

<addressbook>
  <contact name="">
    <name>Sender Lastname002</name>
    <alias>sender.lastname002@domain002</alias>
    <internetalias>reallyobscureuseraliasforinternetuse005234</internetalias>
    <publickey>-----BEGIN PUBLIC KEY-----\n
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC+8G1x7itnnHGlN0kodreKF7cf\n
e8NWiHk8j/tDD49y4m+jsAbpUh0THj2OWTtQhayswUUG7TW4y4qvONdv3zfIOgz7\n
T5aeBHQCXb8Kk54ffFJC+NYiBwwCR9FhxJzy2AXfJweQFXzYJMh+iz7WLebfrmWi\n
rsHJ4ZPtP4SEW4QRWQIDAQAB\n
-----END PUBLIC KEY-----</publickey>

  </contact>
</addressbook>

Recipient properties file, recipient.properties.xml

<properties>
  <addressbookfile>C:\SmartServer_war\RecipientAddressBook.xml</addressbookfile>
  <name>Recipient Lastname001</name>
  <internetalias>reallyobscureuseraliasforinternetuse008621</internetalias>


  <publickey encoding="none">-----BEGIN PUBLIC KEY-----\n
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC+8G1x7itnnHGlN0kodreKF7cf\n
e8NWiHk8j/tDD49y4m+jsAbpUh0THj2OWTtQhayswUUG7TW4y4qvONdv3zfIOgz7\n
T5aeBHQCXb8Kk54ffFJC+NYiBwwCR9FhxJzy2AXfJweQFXzYJMh+iz7WLebfrmWi\n
rsHJ4ZPtP4SEW4QRWQIDAQAB\n
-----END PUBLIC KEY-----
</publickey>
  <privatekey encoding="none">-----BEGIN RSA PRIVATE KEY-----\n
MIICXQIBAAKBgQC+8G1x7itnnHGlN0kodreKF7cfe8NWiHk8j/tDD49y4m+jsAbp\n
Uh0THj2OWTtQhayswUUG7TW4y4qvONdv3zfIOgz7T5aeBHQCXb8Kk54ffFJC+NYi\n
BwwCR9FhxJzy2AXfJweQFXzYJMh+iz7WLebfrmWirsHJ4ZPtP4SEW4QRWQIDAQAB\n
AoGAQlzaDTEUUOyk6yFuAexRGmPnt7+SQu8oBA6e683qs6lT1/aEUG/DvzNRfbox\n
zWjIfTqnPSMQ+utzhjchWbahchIt5QTk2qWIP3zPPaK0eTyvKGrlb65VKkwKfOLx\n
zBWu0wvOzdBnHxx5MlRAYCUa6jlBgIiDsis+GPYtRL7NO8kCQQDoAJpV3aEp/Sdg\n
KkXfdGq47uV9/6dOYcOSYOFU0DwF2M2DXTzFWz/3aUetwMGAzxXkgTKxHvH67FG9\n
evY1hUInAkEA0rB52X2mwkpiAMPJeDxdBO3Ws8DZZotDcCGpWk2c6O+VdvVn2ByQ\n
m1R7vlSPTG0k461vxAua1zWevd7nir3AfwJBAOBKzOqchNXyP6IR3FFm3FKD62x/\n
v8DgdVHLxazYUF8atbz9Y+YtDM9lgf2yL6EXE8RBP7/K+ggrde7otKUTetsCQCcd\n
WlHVcQsrEgzATHwFAOP0X+LZT8Mo4alEppM8S4A3UpFWOCQ4EWS1B5ArVLtEnA+k\n
2FYWLXX8326ynlv0pHMCQQDJwA/yBmYGcJA7qyF1M+VSXDNTg8vwY43kU9LpxfgR\n
sdKpCT7fnyMqj+O84a2gi/X78IBNNR7ZW1t5FIhcRbMn\n
-----END RSA PRIVATE KEY-----</privatekey>
  <encryptionseed>AAAAAAAAAAAAAAAA</encryptionseed>
</properties>

Sending a message

The flow for sending a message is described here

The individual processing steps follow below. The numbers correspond to those on the graphics.

# step 1
Author the message, enclosed within the ‘text’ tags, in Written.xml
Update the timestamp.
The “to” field must match the alias of an entry in the addressbook. The recipients deliberately obscure internet alias is retrieved from the address book using this alias. The obscure internet alias acts as the recipient email address but it’s uniqueness is not assured, unlike a real email address. But that uniqueness is not really required as only the correct recipient is able to decrypt the message anyway.
The “from” field is populated with the sender’s internet alias kept in the sender.properties.xml

During subsequent steps digital signatures are created and validated. To avoid errors it is best to keep line break characters out of xml documents.

Flatten the Written.xml document using the flatten.xsl stylesheet.

java -cp “WEB-INF\lib\xalan.jar” org.apache.xalan.xslt.Process -IN Written.xml -XSL flatten.xsl -OUT Written_flattened.xml

cat Written_flattened.xml > Written.xml

# step 2
# Apply XSLT to create a smart document.
java -cp “WEB-INF\lib\xalan.jar” org.apache.xalan.xslt.Process -IN Written.xml -XSL Written2Transmitted_sd.xsl -OUT Transmitted_sd.xml

# step 3
# Execute SmartDocument

java -cp “..\build\classes;WEB-INF\lib\servlet-api.jar;WEB-INF\classes;WEB-INF\lib\commons-io-2.4.jar;WEB-INF\lib\commons-codec-1.8.jar” com.any14.smartdoc.SmartDocument Transmitted_sd.xml > Transmitted.xml

# step 4
# Send the message to any14.com

curl -d@Transmitted.xml http://dev-smartserver.elasticbeanstalk.com/

# step 5
# Apply XSLT to sent message to create smartdocument used for creating a secure local copy

java -cp “WEB-INF\lib\xalan.jar” org.apache.xalan.xslt.Process -IN Written.xml -XSL Written2SenderStorable_sd.xsl -OUT SenderStorable_sd.xml

# step 6
# Execute SmartDocument

# step 7
# Apply XSLT to the secure local copy to create smartdocument for reading the sent message

java -cp “WEB-INF\lib\xalan.jar” org.apache.xalan.xslt.Process -IN SenderStorable.xml -XSL SenderStorable2SenderReadable_sd.xsl -OUT SenderReadable_sd.xml

# step 8
# Execute SmartDocument

Receiving a message.

The processing flow for receiving a message is described here

And the individual processing steps follow here.

# step 1
# User retrieves recent message

# URL encode the xpath part
# update the timestamp as required (see Written.xml) to get recent messages

curl http://dev-smartserver.elasticbeanstalk.com/app/smartmessage%5Bto/text()=%27reallyobscureuseraliasforinternetuse008621%27%26timestamp/text()%3E=%272014-07-27%2022:05:00%27%5D > Received.xml

# step 2
# Apply XSLT Received2RecipientStorable_sd.xsl

java -cp “WEB-INF\lib\xalan.jar” org.apache.xalan.xslt.Process -IN Received.xml -XSL Received2RecipientStorable_sd.xsl -OUT RecipientStorable_sd.xml

# step 3

# Execute SmartDocument
# note: this step checks the digital signature. If the file has been on a windows system, there is a good chance that carriage return characters have been introduced , in which case the signature validation may fail because of this.

# step 4
check if message is already in “mail database”, and add it if not.

# step 5 – retrieve a meaasge to read it
# Apply XSLT RecipientStorable2RecipientReadable_sd.xsl

java -cp “WEB-INF\lib\xalan.jar” org.apache.xalan.xslt.Process -IN RecipientStorable.xml -XSL RecipientStorable2RecipientReadable_sd.xsl -OUT RecipientReadable_sd.xml

# step 6
# Execute SmartDocument

larsreinertsen

The greatest WordPress.com site in all the land!

Category Archives: any14.com

sailing the data ocean

trivial, secure email application, using command line tools

Sending a message

Receiving a message.