Category Archives: distributed security

Security in distributed systems and for distributed data. Where the data to be protected does not reside on the infrastructure of the data owner.

Secrecy and IP – There is public, there is private and there is secret.

If you ask a networking person what a private (IP-)address is, you get one answer: An IP address which is not routable on the internet. There are some IP ranges which have been set aside for use in private networks and won’t work on the internet. For this reason they are sometimes referred to as “non-routable addresses”. Which is why they tend to be used in movies. Type them into your browser and nothing happens. Unless you are in a local network which have routes for them. Emphasis on local. Addresses that start with “10.” and “192.168.” are the most commonly used, and there are a few others as well. Architects tend to think a private address is one that you own.

But what is a public IP address ? That very much depends on who you ask. Network people will say it is it is an address which in not private. An IT architect will say it is an address which is accessible from outside your network. Security would say one which the public can access.

In some sense they are all correct, and that is why the issue is so confused. Because over all of this reigns security. And more particularly secrecy. There are far too many in the IT business who equate security and secrecy. To be sure the two are not opposing concepts for the most part but they intersect only intermittently – and not often. Some think IP security is mainly about secrecy. If the enemy does not know an address, he can’t attack it.

Ah, would that that were so. This is why the difference between public and private is so important. If an architect thinks that a public address is one that other people know about and a private address is one that is secret, the architect might want to keep the address information secret. Only distributed in secure communication; kept in password protected files and in secure document stores etc.

But how secret can an externally accessible IP address really be.

It is illustrative to think in term of physical addresses: A person lives on a secure address. But the house is still there; Numbered sequentially as you like; and the street is public. Anyone can drive to the house at any time. How secure is that? Let’s further suppose that those looking for the occupant of the secure address know that the house has been provided by a company that owns a particular street (public IP range discoverable in DNS). It is a fair guess that the right house is on that street. What does that do to security? If the house is a fortress, it doesn’t matter. But if you base your house security on it being undiscoverable, nothing good.

NAT-ing does nothing useful here. It is like having all your mail sent to the post office and have the post office forward it to your house. Someone can still mail you a remotely detonated bomb (malware). The post office will forward it just the same. (wait a day; trigger it; look for smoke; and you have the right house)

But there is another way: A private road (non-routable/private addresses) cut off from the public road system; you have to be authorized and be lifted in. That is a secure address. Not a full replacement for security at the address of course – you still lock the doors – but the address information is secure as in scanning will not discover it. No looking inside mailboxes on a street you can’t get to.

How to achieve this happy state of affairs. Enter the muddle.

Virtual Private Networks. With “private” here referring both to private addresses as defined above (non-routable) and as in you own it. VPNs are sometimes referred to as encrypted tunnels. Typically based on IPSec but there are other possibilities. Any IP address can be used in a VPN but using a VPN you can have your non-routable “10-dot” address be accessible from anywhere, even over the internet.

And this is done. What strikes me as peculiar is when VPN is used with public/routable addresses. The address is still discoverable, because it is public and no amount of secrecy will change that. The house is on a public street – no matter that the occupant is not listed in the phone book and has his mail sent to the post office. It will be a part of a range that has some other member of it in a public DNS. Therefore it is discoverable through scanning. This is also why services should not run on default ports. And all services running on default port should be disabled as far as possible. Knowing an IP is only half the battle, you still need the right port if you are going to attack something.

Add VPN to the public address and unknown traffic will be prevented from reaching it. Which negates any need to keep it secret – not that you could even if you wanted too. Which leaves no role for secrecy when it comes to IP addresses.

Why even use a public address if you are going to use VPN? They are neither free nor plentiful. Well, you shouldn’t. But if you do, it must be for reasons of convenience. They are unique after all. There is no need to agree a private/non-routable address with the counterparty so as not to cause a conflict with something else in their network. Adding NAT-ing here is just silly as it simply adds confusion and does nothing for security. With one exception: if there is a failure to agree on a single mutually acceptable private address among multiple parties. If there is a hold out, there might have to be NAT-ing between that party and the others.

No, a VPN tunnel should be from one private, non-routable address to another. Not that this needs to be point-to-point. You can route all kinds of traffic though a single VPN tunnel. Your routing tables control that.

Server A has address 10.1.0.10 and sends traffic to Server B at 10.2.0.10 over a VPN. And vice versa. They belong to two separate organization and separate private networks.

The VPN tunnel has openings 10.1.0.1 and 10.2.0.1 respectively. Server A routes traffic bound for 10.2.0.10, to 10.1.0.1. Server B routes traffic bound for 10.1.0.10, to 10.2.0.1.

All of these addresses could be freely published and it would do any external attacker no good at all. Security without secrecy.

Token-in-token, Agent-Principal model of authorization delegation

PAML tokens create portable authorization where the Enforcement point need not know either the Principal making the request or the Owner authorized to grant the request. But the Principal/user must be in direct contact with the enforcement point to establish, through a cryptographic handshake, ownership of the PAML token being used. In many scenarios this direct connection is undesirable, even impossible. The Principal can not have a direct connection to the enforcement point.
A medical records repository is a good example. Security rules prohibit direct access over the internet to such a repository. Intermediary entities, application servers, firewall and the like, are mandated gateways.
In effect someone or something, must act on behalf of the Principal with the Enforcement point: An Agent.

This can be effected by placing one PAML token inside another. This token-in-token functionality is described in detail below. Any number of tokens can be embedded inside one another. In some ways like a certificate chain.
As with a standard Agent-Principal model the Principal trusts the Agent to act in the way the Principal desires. But with PAML token the Principal can narrow the Agents latitude to the extent desired.

Definition of terms

The Principal. This is the person, or entity, on whose behalf the request is undertaken.

Agent. The person or entity carrying out the request task on behalf of the Principal.

Owner. The person or entity that decides if the Principal is entitled to have the request(s) undertaken.

Enforcer. Person or entity in control of the resource and handles the request from the Agent. The Enforcer accepts the PAML token for authorization decisions.

Here is a schematic of the flow

alt text

Agent-Principal use of PAML tokens

 

Explanation of steps
Step 1 Principal prepares a list of request, or logical super-set of requests, in an XML format document; and signs this list with the Principal’s own private key. The logical super-sets can include rules according to which the requests made by the agent should be evaluated. The Agent can only carry out request on this list or ones contained in super-sets on this list. The XML document can include XSLT operations to be applied at enforcement time to the request made by the Agent. With this XSLT the Principal can enforce additional limitations (such as timing or checking token revocation list) on the requests the Agent is making.

Creating (pseudo XML document)

<principal.signature><request.list/></principal.signature>

Using the private key used to sign the request list document and the public key that belong to it: engage in cryptographic handshake with Agent and through this, establish to the Agent’s satisfaction that the Principal is in possession of the private key used to sign the request list.
Pass the signed list to the Agent.

Step 2 Agent attaches own usage conditions to Principal’s signed request.

Creating

<principal.signature><request.list/></principal.signature><agents.usage.conditions/>

And sign with the Agent’s own private key.
Creating

<agent.signature><principal.signature><request.list/></principal.signature><agents.usage.conditions/></agent.signature>

Using the private key used to sign the document and the public key that belong to it: engage in cryptographic handshake with the Owner and through this, establish to the Owner’s satisfaction that the Agent is in possession of the private key used to sign the document.
Pass the signed document to the Owner.

Step 3
Owner attaches own usage conditions to Agent’s signed request.

Creating

<agent.signature><principal.signature><request.list/></principal.signature><agents.usage.conditions/></agent.signature><owners.usage.conditions/>

And sign with the Owners own private key. This is the same private key used to sign the resource the PAML token governs access to.

Creating

<owner.signature><agent.signature><principal.signature><request.list/></principal.signature><agents.usage.conditions/></agent.signature><owners.usage.conditions/></owner.signature>

The PAML token is now completed.
Return the PAML to the Agent.

Step 4
The Agent now has a PAML token providing authorized access to resources signed with the same private key that signed the token. The PAML token can be reused to the limits specified in the owner’s usage conditions. Typically these would at least include validity time periods.
The Agent can now make request to the Enforcer and have them be authorized by the token. The requests are on behalf of the Principal but do not have to be directly from the Principal.

The Agent engages in a cryptographic handshake with the Enforcer using the private key used to sign the document from the Principal and the public key that belong to it: engage in cryptographic handshake with the Enforcer and through this, establish to the Enforcer’s satisfaction that the Agent is in possession of the private key used to create agent signature in the PAML token document.
Passed one or more PAML tokens to the Enforcer.
Pass a request to the Enforcer.

Step 5

The Enforcer examines the PAML tokens it has received. The tokens are checked in the following way and order:
XML validation. Those containing invalid XML are discarded.
XML schema validation. Those containing improper XML are discarded.
Agents signature validation: Using the public key of the Agent established in the cryptographic handshake the agent signature in the token validated. Tokens failing this validation are discarded.
Remaining tokens are tokens the Agent is properly entitled to use.

Examine the request and match it to the request list part of the remaining PAML tokens.
Those token that do not govern the request are discarded. Meaning that the request list in the token(s) do(es) not have at least one match for the request.
The remaining PAML tokens, if any, both belong to the Agent and govern the request the Agent is making.
The Enforcer examines the resource the request is for and what, if any digital signature it bears. Only if the resource bears a signature that can be validated with the same public key that can also validate the owners signature on the PAML token, is the resource considered “in scope” for the request. These two signature validation are carried out using the public key of the owner included with owner’s signature on the PAML token. If both signatures can be successfully validated the request is authorized.

In step 5 we get the desired situation that a completely stand-alone enforcement point can make authorization decisions for agents acting on behalf of someone else, without having to know the identity of either of the parties. This gives obvious benefits in reduced complexity and  ease of deployment.

sailing the data ocean

What if there was a way for access to data could be authorized everywhere. If you were authorized to access a piece of data you could get access to it wherever it happened to be located.

This is not the way things work at the moment for sure, but if it could be made to work in a convenient way, what should it be like ?

When the first web browser arose some 20+ years ago. Static html pages and other media and document files where available by calling a URL over the HTTP protocol. Security was added – at first pretty coarse grained: If you were logged in you could access pretty much anything. It got better.
Comprehensive tools became available to centrally manage all web access to any document, with the finest granularity. The writing of the rules of who should access to do what, when and where, could be delegated out to those who actually were in a position to know.
But crucially, these tools could only manage access to stuff directly under their control. Often operating in a reverse-proxy mode intercepting HTTP traffic. APIs were available through which other applications could tap into them to take advantage of access control rules contained in them, to do their own authorization. In this way the data under the control of a unified set of access control rules could be made corporate wide. Access to all of a data in a corporation being governed by rules maintained in one place. Everyone would play together in the same data security pool.
In practice this never happened. (re-)Writing applications to take advantage of the API of the chosen security software platform , was too expensive. Other tools emerged to export the security rules from one software platform to another, leaving them to do their own enforcement through their own rule infrastructures. This didn’t work very well because it was too complicated. Rules are fundamentally about meaning, and meaning doesn’t translate easily. Never the less this was an attempt to federate authorization.

Data protected by the same access control rule infrastructure is part of the same pool. A database is a single pool. It has its own internal security governing access to individual pieces of data contained in it, but has no reach outside. The database maintains it’s own list of who gets to access which column in what tables.
A server has it’s own internal arrangement for governing access to the data in its own file systems. It may also have access remote file systems. Some remote file systems would be on other servers, which would govern access (NFS, FTP, Samba etc.) and would therefore not be part of the server’s own pool.

If authorizations could be federated between pools all data would exist in one big virtual pool.
A virtual pool made of multiple physical pools; individual databases, file servers etc. At present this is difficult as there may be user federation between some data pools, but each pool has it’s one authorization, it’s own way to enforce access rules. The rules in one are not known, or directly enforceable in another. There is no federated authorization.

Lets further suppose that any piece of data in this virtual data pool, data ocean really, is accessible over TCP with a URI. The URI may have various formats depending on what type of physical pool is being addressed.
For example, this would be the syntax of an URI accessing a directory (LDAP) store

ldap[s]://hostname:port/base_dn?attributes?scope?filter

And this to access a individual file, using HTTP(S)

https://host:port/path

Access to one of the secure web reverse-proxies mention above, would look like this too.

The would be many others. Note that the username and password does not appear. There would not be any prompting for this information either.
Access control would be through PAML tokens, passed in the headers. A SSL handshake would take place to establish the requesting entity’s authorization for the tokens presented.
All physical pools are defined by the entity that control access to it, and all of these entities, be they LDAP server and file/web server in the URI examples above must be equipped to handle PAML tokens to verify the authorization for the request. Through the acceptance of these PAML token the pools together form a virtual data ocean. Any application can call on data anywhere else and present PAML tokens for authorization.

This leave quite a bit of scope for application architecture. The use of a PAML token require access to the private key of the user to which the PAML token was issued. Which means that if a user is engaged in a transaction with an application and this application needs access to data kept somewhere else on behalf of the user, the application can only present its own PAML token, not forward those it has received from the users. The user must at a minimum contact this other data store directly and engage in a SSL handshake. This way the user’s ownership of the public key is established for the benefit of the data store. The application can then pass the PAML token received from the user on to the data store and the store would now know that the PAML tokens are OK to use; or the user could make the data retrieval directly and pass the data to the application that needs it. Sort of like a data federation.

Note that PAML token are tied to data, not any particular host environment. Among other things this means that the requesting client may send the server a considerable number of tokens in order to establish authorization for all required data. The server will grant the union of all these tokens.

IT security shapes business models

text slightly reworked from talk given to OIC, Oslo Dec. 2014

In his article The Nature of The Firm, Ronald Coase proposed that a firm forms and grow while transaction cost inside the firm are lower than outside it. I.e. a firm can do something cheaper and better in-house than going out in the marketplace for it.

OK, that’s a company, but what about their data ? It seems that something along those lines are going on there too.

Most of us have come across this in the form of federated login. You’re about to log in at a web application and are given the option of login in via Facebook. Or Twitter. OK, what happens if you click on the Facebook button ? Clearly something, since you enter the application in question and are now known as yourself.
This is where federated security comes in. If you where already logged in to Facebook, a message was sent to Facebook were your valid logon session was used to prepare a special access token. A token that the new application could use to establish a new session for you. If you where not already logged in to Facebook, you could log in now and the same thing would happen.
In any case you log in only to Facebook, and other web sites take advantage of this to log you in to their site too.
Acting in this capacity Facebook is called an Identity Provider. And no, that is NOT a deliberate pun. A company, or application that uses such a federated login, is in the jargon called a service provider.

Returning to Coase again. The application can have it’s own login and user database. Doing it in-house. Or you can go out in the market and contract for this service, as he suggests. Allowing login through Facebook.
OK, how much would such an external login service cost in the market? As it turns out, not very much.
For one thing, the marginal costs are low. Once you already have the users and their passwords in your database; adding a token exchange where by other applications can send their users over for authentication, is quite cheap.
Not only is it cheap to allow third parties use you login process, there is also revenue in it. You can now guarantee that the users have the same userid across platforms. All kinds of people would be very interested in that, and pay good money to ensure that it happened. And not just the NSA. Advertising and other businesses who make their money by analyzing consumer behavior likes federated login a whole lot. If you allow other companies to login their users via your platform, you control the origin of this data.

Considering the economies of scale being so much in favor of the federated login and those that make it happen. Why aren’t all logins federated ?

Well, more and more are.
But the savings may not be that large on the service provider side. The in-house part of Coases analysis. Just because you leave the authentication of your users to someone else doesn’t mean you can then forget about them. There is also that bit about authorization. What are people allowed to do. From a security point of view the whole point of identifying the users in the first place is so that we then can then look at our rules for what the user should be permitted. If we don’t know who they are we don’t know what they are allowed to do.

So you still end up needing a user database. And a database of rules for what the users are allowed to do. And maintain those things. Particularly that last bit can be complicated and expensive. Once you have all those things, handling the login isn’t that big of a deal.

From a revenue point of view things are a bit different. Identifying the users has it’s own value – selling information about the user to third parties.
You could limit yourself to just logging the user inn, so that you may accurately track him, but not bother about doing any authorization.

And this is what we see a lot of. Plenty of media sites accept federated login through the likes of Twitter and Facebook.

They only care about getting you name accurately. Their revenue comes from knowing it.
There is no fine grained access control or authorization going on. It is not worth it to them. Disqus.com is an example of this. And your email address is becoming your universal user id another.

Clearly Coase’s balance between transaction costs in the market place and in-house, are still very much in play.

This suggests a possible future.
If we think about a corporation as being fundamentally an agglomeration of data and business practices. Where interaction between businesses, trade, is an interaction between these agglomerations.
Returning to Coase’s starting question, the boundary between a corporation and the market and why they are where they are.
What defines the boundary of these data agglomerations? The authorization rules. The data might be located anywhere. At Dropbox even. But if you write the rules governing access to it, that data is part of your agglomeration, wherever it is. If someone wants to interact with your data or your business processes, you must write the rules to allow it to happen. Your rules map out your agglomeration. Outside the reach of your rules is outside your boundary.

Which sketches out the premise that the limit on the size and shape of a corporation is really a limit of information technology. Specifically the limit of authorization technology.

This is all very well, but what does that look like in real life. What are the business strategic implications of such limits. If they even exist.

There are grounds for suspecting that they do.

We have technical standards for federated login. Therefore we have the Facebook login example. A user known from other places on the web is more valuable on the consumer profiling market than one know just to yourself. Both parties gain. The consumer not so much.

But we do not have standards for federated authorization, much less any off-the-shelf technologies for it.

What should we expect to see if federated authorization were possible?

Truly distributed content for one thing.
At the moment digital content is licensed, which is another way of saying it is rented. Meaning that the right to authorize access to it has been delegated to the licensee for a limited period of time. And the licensee writes the authorization rules for the duration. A temporary data transfer from one agglomeration to another. The data is not really distributed, certainly not by you. Someone else distributes it on their own behalf.
This disintermediation is risky for both parties. If the owner overcharge for the license no one will buy one. Undercharge and you leave money on the table. For the licensee the risk are the reverse. Overpay and you lose money. This makes the distribution process contractually complicated. Made so in no small measure because there is no way to federate authorization. These risks mean that there will be much less of this business than there potentially could be.

So the idea that buying a ticket to a movie would get you a free repeat on Netflix is, as things currently stand, out of the question. Despite the obvious advantages for all concerned. The movie companies and Netflix are separate agglomerations.

And it’s not just media. All kinds of content. Medical records. It is impossible to have fine grained access control to the individual data items in such a record after it has been compiled. Therefore its distribution is exceedingly complex and cumbersome. If you can see any part, you can see the whole. If access to an individual item could be controlled wherever the document was, it would be simple to distribute it. Those that needed to, and had authorization to see an individual data field, could see it. Without always having to be cleared to see the whole thing. As it is,
we now have checkout clerks in pharmacies having access to much your medical records because they need to verify one small item in them.

Federated Authorization

For a while now the use of federated userids has been the norm. A site called the Service Provider, SP, has entered into agreement with another, called the Identity Provider, IP that it will trust the userid the IP sends it as being properly authenticated and accurately identifies the user.
There are a number of way this can be done; the passing of a SAML token from the IP to the SP is a popular way. Where the SAML token contains the userid and the SP trusts this because the token is signed by the IP.

So much for users and their userids, but what about what they have access to; The authorization.
Userids are tied to revenue in significant ways. In businesses deriving revenue from gathering information about the users, the useris is clearly of primary concern. More generally in business applications it is not the userid itself that is central, but what the user has access to and how to execute that access control.

In a standard access control deployment scenario with federated users. For example the user login somewhere else (the IP) and click a link to (return to ) the application in question, with the userid being passed along. This is a federated login. The application (the SP) still examines the userid and refers to its own access control infrastructure for what the user is authorized to do. Could this step be federated too ? Should it ?
Like a login and authentication infrastructure costs money, so does maintaining an access control infrastructure. Federated login cuts the costs of the first, so can Federated Authorization, FA, cut the second.
Form the cost point of view FA makes sense.

Can it be done and does it make sense from a policy and governance point of view ?. It is one thing to let someone else identify the user on your behalf. Digital certificates have been around for a while so the issues surrounding outsourcing the work of identify users are not new. And the risks are largely accepted. But authorization cuts close to the essentials of governance: what are people permitted to do. True, if the user is misidentified having a thorough access control system does not make much difference.
One significant issue is related to changes. A user ID doesn’t change much, if at all. What a user has access to, does. Sometimes great urgency. Having direct control of authorization is then desirable.

The day-to-day problem with access control is not primarily the enforcement, though that can be tricky enough. It is the creation of and updates to policy. The rules governing the access. Who gets to decide and how can those decisions be captured in an enforceable way.

another Silk Road takedown

here we go again… The powers that be have take down some evil doers hiding behind TOR. I’m must ask pardon for being a bit jaundiced about the hole thing.
Clearly there are people doing undesirable things taking advantage of the protection offered by TOR. but TOR has a legitimate purpose and the support behind TOR is impeccable. But TOR is not perfect, specifically it is not fool proof. User error compromises it.
But I’m wondering is there is another way for the vendors on Silk Road to conduct their business. Leaving aside the problems of payment for the moment: BitCoin has it’s own weaknesses.
Concentrating merely on how two parties can get in touch with each other. Were A is looking for something that B has to offer, put very generically.

If my understanding of the attack against Silk Road is correct: It is based on having control over a significant portion of the onion routers. Sufficient for being able to establish a pattern of traffic from yourself to a particular .onion address.
Each request packet is routed every which way by the TOR protocol. But with control of enough routing nodes a pattern will never the less emerge since the request packets must eventually end up in the same place: Where the concealed service is hosted.
This last bit is true even without implementation errors. DNS leakage and the like.
What if the hidden service is not hosted in just one particular server, but on many different ones. And using the service will involve traffic to many different locations. Is this possible and whould it make a service concealed behing a onion routing scheme undetectable ?
That depends. If the hidden service is hosted on a fixed set of servers rather than just one, there is no real difference. Just that more traffic now needs to be analysed to be able to pin-point them. This could be counteracted by moving the service often, but that is a defence mechanism independent of how many host the hidden service is using. And controlling more onion routers will help the tracker.

Trusted VPNs is of course an option. Where both the client and the hidden service use VPN to tunnel to some part of the internet. The tracker is then left with finding a proxy rather than the real host. It is still possible that the tracker might use traffic analysis to get from the proxy to the real host. As long as the service stays put long enough.

This suggests a possible course of action. Can the hidden service be dynamically located ? Perhaps even randomly.
I think I’ll work on that. A facinating challenge. Watch this space.

accepting PAML tokens

PAML tokens where developed to address the needs of a distributed environment. Providing the ability to make decisions at data access enforcement points without having to contact other parties.
This sounds dandy, but how exactly is it done ?

The high-level process is simple enough.

The devil is luring in the usual place.

The detail here is the digital signature on the resource item in question.
Any data item can be digitally signed, but it the data item is small the signature can be larger than the original data. There is scope for optimization here.
In the PAML token sample the digital signatures are the larger part of the total data. The balance will not be so skewed in more typical examples. But the fact remains that the digital signature in XML documents is very verbose.
This need not be the case where user data is being stored. Assuming the data item is database column with metadata, timestamps and such tings in the other columns. Additional columns can be added to contain only the relevant bits about the signature: Algorithm, Digest, Signature value. This would not take up nearly as much space and would be manageable. For optimization it would be helpful to also store the owners public key (not certificate). This would speed up processioning. When a user’s PAML Token has been accepted and the user’s request has been determined to be permitted by the token, the owners public key is known from the token. This key can be used as a mask on the data, added to SQL statements to further narrow the search in the database and exclude all rows containing data not governed by the PAML token. The user’s request is then applied to the remaining.

Big Personal Data

Reading this article in a Blog on New York Times today I kept thinking about the what was not being said in it. Data governance.
The assumption, not doubt valid, is that ever larger pools of data will be amassed; and suitable analyzed, solutions will be found to existing problems and fantastic new possibilities will emerge.
I don’t doubt the potential in “Big Data” and look forward to its exploitation with interest. But also with some trepidation. With great power come great scope for abuse. This is as true for technology as it is for people. Necessarily so, since it is people how design and use the technology.
The article mentions car sensor data and its collection. But what is this data and who get’s to use it ?
Going on to mention ever higher transfer speeds allowing larger data quantities to exchanged ever faster.

The term Data Lake is used to describe GE accumulation of sensor data. Siting that it permitted speeding up GE’s error detection by a factor of 2000.

With ever larger quantities gathered and ever faster transfer speeds and distributed analytical tools all the worlds data eventually becomes available in a Data Sea, ready for analysis.

The article argues that this is not just technically possible but also both the trend and desirable. The first two are clearly correct. The problems with different data structures and qualities are surmountable. As for whether or not it is desirable the issues are less clear.
That there are advantages to be had is obvious. It is also probable that the advantages are great enough to justify the expense involved. Indeed it would not happen in the commercial world unless there was money to be made. Governments have other objectives, social control including law enforcement, where the expense is not a primary concern, only technical feasibility.

In engineering technical feasibility is a function of resources expended. And the Big Data field is certainly seeing a great deal of resources expended on it, whatever the motivation. So what is possible in future will certainly expand from what is possible today.

Data governance has become a political issue of late. The relentless surveillance by governments of their citizens is becoming more widely known. Not all governments have the resources of the US or the determination of Chinese. But that is a temporary relief. The capabilities will expand and the costs will decline. In due course everyone can aspire to total surveillance.
At present only the richest and most powerful can access the whole data sea. Others can only access the part of it they gathered themselves. This article implies that limitation will fall away and everything will be accessible to those that can pay. But who gets the money? For now it has been those who accumulate the data that get the money. They incur the expenses; servers, storage and bandwidth cost money. The source of the data gets nothing. Certainly no money but in some cases a service. Social media run on expensive server platforms but the user can use them “for free” in exchange with surrendering all their data and any other information the platform can gather about them. Many consider this fair exchange. Even if this is so, it still in the users interest to control where the data is going, who gets to use it and for what. The stories of social media posts that have circulated outside their intended target audience and caused embarrasment are legion. The columnist Kathleen Parker advocated self-sensorship to a degree inconsistent with civilization.
It is unclear whether she honestly meant what she wrote or had simply disengaged her mental faculties before writing the piece. Bill Mahers response was withering and to the point. We can’t live in a world where the only privacy we have is inside our heads. Privacy matters also for those less exalted among us. Scanning social media before a hire is now common practice. And individuals go not get to decide what they need to “hide” – anything that may cause offense or raise the slightest question can cause later inconvenience or worse. Made even more intractable by ever changing social conventions.
Expect to see much, much more on this subject.

Some gatherers of user data have policies in place and tell the user explicitly that while they gather as much data as possible they do not pass it on to any one else. Or if they do, it is in some lesser, anonymized form.

That data is passed along at all suggest convincingly that the data has commercial value. As interconnections improve and data sales increase it is also clear that the source of this data are being underpaid; the data gatherer can increase their revenue per data item without incurring any additional source costs.

Persons who are the source of this data then have both a financial and a data governance issue (read privacy) at stake.

Social media users voluntarily submit to being guinea pigs to marketers. Their data has a clear commercial value and this is what they use to pay for the service.

In due course platforms will appear that also allow users to pay in other forms than just with their own personal information and data they provide themselves. “Pay small monthly fee and we will not sell your emails to your local grocery chain”.
Pay-for-service is available in social media, particularly in the more specialized subsets, like dating, but it is always either a pay or spy business model. Never a commingling of the two on the same platform. This is in most cases driven by technical considerations. It can be tricky to adequately safeguard the user data that is not available for resale. But see

Internet of Things is the next step. How are we going to handle that data. The Big Data providers have the datastore ready to take on the data and the tools to analyze it. But where are the tools to safeguard the individuals ?

computing when you don’t trust the platform

This appears to be virtually contradiction in terms. In IT security racket we have always stressed the importance of having physical control of the hardware. Anyone with physical access to the device could gain root access to the system and from then on do pretty much anything that wanted.
By extension we could only trust the application, and the safety of our data contained in it, if we also trusted the physical safety of hardware.
The Microsoft verdict is interesting in this regard. If the cloud vendor is US company, a US company own the hardware and it is thus subject to US jurisdiction regardless of there it is located. At the moment we do not know if this verdict will survive a supreme court appeal. But in any eventuality the conundrum remains: Can we have trusted computing on a untrusted computing platform ?

I’m going to try to demonstrate that this is possible for a server application.

Clearly a cloud based web application constructed to work on an untrusted platform would be very different in it’s internal workings from how things are generally done now.
The working model that I propose to pursue will be a web service with thick clients. The web service will run on a cloud vendor subject to US jurisdiction, AWS. I will demonstrate that the sensitive application data will sit on the AWS in encrypted form, at no time be decrypted on the AWS nor have the encryption keys at any point be on the AWS in clear.
The thick client will be an Android app.

In the event that I am successful using a thick client (app), I will attempt the same using a thin client (web browser).

How to secure it on the Internet of Things

The internet of things is not a new idea but in the hype curve it is now definitely on the upswing. Disillusionment should be setting in soon, before the real delivery of value gets started.

I  had an amusing conversation with the CEO of a startup in the business of providing software for monitoring “things” on the Internet of Things. Amusing in that whenever I asked him about how access to these devises was secured, he said that you didn’t access them directly, only a virtual representation.  I then followed up with “so they are not connected to the internet then ?” to which he answered yes, they were. I returned to my original question. Repeat. We went though this cycle about 4 or five times. We had a pretty good bit going there; should have taken it on the road. As a CEO of a startup he could have used the money I’m sure. I am also sure that having your security model be a joke on the standup-circuit would perhaps be ill advised.

That still leaves the larger question. What should be security model be for the Internet of Things.

In some sense we have had IoT for a long time already. But the “T” in IoT are not the PCs and servers we are used to.  These new devices will not be running a full stack of software. They won’t  be able to and even if they were (computing resources gets ever cheaper) it would be hugely wasteful. And a management nightmare too.

Though I’m sure many software vendors will emerge to help us with it. And Microsoft would be perfectly happy to sell a copy of Windows to every fridge and toaster built.

No, these devices will have a simplified software stack. As simple as possible. Requires less hardware, and less is cheaper. Perhaps cheaper by only a few cents, or less, but when you number your devices in the billions, every little bit adds of pretty fast. Simple is also easier to maintain and leaves less room for error (read: bugs).

Considering only the security side of things. What are the minimum security features required ?

Privacy: The devices have to be able communicate privately – which means encryption.

Integrity: The devices have to be able communicate reliably. The data streams must not be tampered with. This ties in with encryption in the TSL/SSL protocols.

Authorization: If the device is able to receive and act on requests , there has to be a way to check the authorization for those requests.

The need to authentication is typically driven by the need for authorization – first you establish who somebody is, then what they are allowed to do – but with PAML tokens this is not so. Authentication is therefore optional, depending on security protocol.