Analysis of UK General Election Betting – 2 May 2015

Introduction

Previously I have analysed the betting patterns around the Scottish Independence referendum, and found some significant trends (here). So, as we near the final conclusions of the UK election, we will see if we can use some of the pointers from the Scottish Independence referendum to highlight possible signposts. One thing that was highlighted in the analysis of the referendum is that the bookies actually predicted the result, and give pointers to the geographical votes, at an earlier stage than many of the analysts.

Here is a summary of the highlights on 2 May 2015:

• Next Prime Minister: Ed Miliband (4/5), followed by David Cameron (Evens)
• Most seats: Conservatives (1/5), followed by Labour (40/1)
• Likely result: No overall majority (1/10), followed by Conservative majority (6/1).
• Most active bet: Number of seats for UKIP (1,197 changes in 2015).
• Conservative seats (average): 286.6
• Labour seats (average): 268.8
• SNP seats (average): 51.5
• Liberal Democrats seats (average): 26.8
• Plaid Cymru seats (average): 3.5
• UKIP seats (average): 2.6
• SNP to win all the seats in Scotland: 6/1
• SNP to gain over 50% of the vote in Scotland: 2/15

Outline of odds

In the UK election poll, there are four horses in the race, so over the next few months we’ll be analysing the key turning points in the odds. The way that odds are normally defined is the fraction which defines the return, so Evens is 1/1, where for every £1 bet, you will get £1 back in addition to your stake (so you get £2). If the odds are 2/1 (2-to-1 against), you get £2 back plus your stake (so will get £3 on a win). For 1/2 (or 2-to-1 on), you get half your money back, and you’ll get £1.50 on a win. These types of odds are known as fractional odds, where the value defines the fraction for your payback. The multiplier, though, does not show your stake coming back to you, so decimal odds are used to represent this, and defines a value which is multiplied to the stake to give the winning amount (basically just the fractional odds plus 1, and then represented as a decimal value).

The factional odds value of Evens gives a decimal odds value of 2 (where you get £2 back for a £1 stake), and 2/1 (2-to-1 against) gives 3.0, while 1/2 (2-to-1 on) is 1.5. In terms of roulette, Evens would define the odds for a bet of Red against Black (as each are equally probable). In roulette, though, the odds are slightly biased against the player for a Red v Black bet, as 0 changes the odds in favour of the casino. For betting, overall, bookmakers try to analyse the correct odds so that they have attractive ones (if they want to take the best), against others. If they take too much of a risk, they will lose, so their odds around the election vote should be fairly representatives of the demand around bets, and the current sentiment around the debate.

Current betting

As of 2 May 2015, the average decimal bets are given in Table 1. It can be seen that the Conservative are 5-1 on winning most seats with Labour sitting at 10/3.

Table 1: Current average odds

Labour	4.3 (approx 10/3)
Conservative	1.2 (approx 1/5)
UKIP	413 (412/1)
Liberal	929 (928/1)

Majority or not?

In terms of a majority or not, the bookies are predicting that there is not likely to an overall majority, and this sits at 1/10, which is the kind of odds you would struggle to get even if Barcelona where playing Luton Town, at home. The Conservative party are the closed to an overall majority, but still sit out at 6/1, with Labour at 40/1.

No Overall Majority	1.1 (1/10)
Conservative Majority	7 (6/1)
Labour Majority	41 (40/1)
UKIP Majority	241 (240/1)
Any Other Party Majority	583 (582/1)
Green Majority	1001 (1000/1)
Liberal Democrat Majority	819.2 (818/1)

Trends

From a starting point of being equal for betting around Sept 2014, the Labour party has generally drifted out from 1.8 (4/5) at the start of Jan 2015 to 4.4 (10/3) on 2 May 2015 (Figure 1), while the Conservatives have moved from 2 at the start of Jan 2015 to 1.2 on 2 May 2015 (Figure 2).

Figure 1: Analysis of the Labour’s odds for winning most seats (6 Aug to 27 Oct 2015)

Figure 2: Analysis of the Conservative’s odds for winning most seats Dec 2014 to 2 May 2015)

Generally the bookies are seeing a move towards a certain majority, and can see this best if we plot the average odds from the start of the year to 2 May (Figure 3). Generally the Labour odds keep drifting by the day, and the Conservatives are coming in. At the start of the year, Labour were favourites to gain most seats.

Figure 3: Average of odds for Labour and Conservatives (Jan – 2 May 2015)

Predicted number of seats

While the Conservatives are favourites for the majority of the seats, the favourite for the number of seats they will gain has not varied much of the past few months (Figure 4), with a current estimation at 286.5 seats as apposed to 282.8 in Jan 2015. Labour, though, have slipped for the number of seat from 287.5 to 268.8 (Figure 5). We can see two significant events in the predicted number of seats for Labour, and which occurred around the Leader’s Debate (26 March 2015 and 2 April 2015). Unfortunately for Labour, the bookie reckonned that these were only small blips, and the number of seats quickly settled back where they had been before the debate.

Figure 4: Predicted number of seats for The Conservatives (Jan 2015-May 2015)

Figure 5: Predicted number of seats for Labour (Jan 2015-May 2015)

In terms of UKIP, the predicted number of seats from the bookies around the start of the year was 6.5, but this has fallen to a 2.6 seat average prediction (Figure 6). UKIP are also set at 9 (8/1) to not gain a seat, so look a strong bet to get at least one seat. The number they will end up with is currently a hot bet.

Figure 6: Predicted number of seats for UKIP (Jan 2015-2 May 2015)

For Liberal Democrats the trend for seats from betting has not shown much change with a variation from a starting point of 27.5 to a current threshold average of 28.8 seats (Figure 7).

Figure 7: Predicted number of seats for UKIP (Jan 2015-2 May 2015)

So the current standing from the bookies for the average threshold for seats is:

• Conservative seats (average): 282.8
• Labour seats (average): 268.8
• SNP seats (average): 51.5
• Liberal Democrat seats (average): 28.8
• Plaid Cymru seats (average): 3.5
• UKIP seats (average): 2.6
• Greens seats (average): 0.5

The Next Prime Minister?

In terms of the average betting on 2 May 2015, the next prime minister it is very much looking like it is level-pegging between Ed Miliband and David Cameron, with Ed Miliband slight more of a favourite at 4/5, with David Cameron on Evens:

• Ed Miliband 1.8 (4/5)
• David Cameron 2 (Evens)
• Caroline Spelman 51 (50/1)
• Boris Johnson 61.9 (61/1)
• Tristram Hunt 67 (66/1)

David Davis, Grant Shapps, Jeremy Hunt, Justine Greening, Liam Fox. Rachael Reeves and Theresa May sit behind these candidates at an average of around 100/1. At the start of the year, David Cameron was well in the lead in terms of odds (at 1.8), with Ed Miliband on 2.3, but over the past few days Ed has overtaken David, but both the odds show them moving closer together (Figure 8).

Figure 8: Average odds for who will be Prime Minister (Jan 2015-2 May 2015)

Scottish Analysis

In Scotland, the bookies are currently pitching +51.5 seats as a key breakpoint for SNP seats. If we look at the trend, the bookies have been moving up the breakpoint bet for the number of seats they will gain from +6.5 in April 2014 to the current value of +51.5, and have increased by 43 seats (Figure 9).

30 Apr 2014: +6.5
26 Aug 2014: +7.5
18 Sept 2014: Referendum vote
22 Sept 2014: +8.5
10 Oct 2014: +10.5
19 Oct: +11.5
27 Oct: +12.5
2 May: +51.5

Figure 9: Predicted number of seats for SNP (Sept 2014- 2 May 2015)

The bookies are thus predicting that the SNP will gain 51.5 out of 59 seats available. Only one bookie is laying bets on them winning all the seats, and it is pitched at 6 (5/1). In terms of SNP winning the majority of the seats in Scotland, SNP sit at 1.01 (1/100), with Labour at 51 (50/1). Another bet sees SNP running at 1.13 (2/15) for over 50% of the vote in Scotland, and where Labour at 5 (4/1) to not gain any seats in Scotland. The Scottish Conservatives are currently 2 (Evens) to gain zero seats.

Most activity on betting?

Punters will generally be looking for a good place for a bet, and aim to get in early before any movements to bring the odds in. From the start of the year, the number of bet odd changes have been:

UKIP 1,197 bet changes
SNP 212 bet changes
Liberal Democrats 50 bet changes
Liberal 48 bet changes
Labour 47 bet changes
Conservatives 30 bet changes

It can be see that perhaps the number of seats that UKIP will gain is the most variable factor in the betting dynamics.

Turn-out?

The predicted current turnout threshold is 68.5%, and which has remained almost static since the start of the year.

What is most likely?

So, in the end, what are the bookies predicting for the end result? The favourite is a Labour minority at 2.7 (17/10), followed at 10/3 by another Conservative/Liberal Democrat coalition at 10.3:

Labour Minority                                2.7 (17/10)
Conservative/Lib Dem Coalition   4.4 (10/3)
Conservative Minority                     4.9 (4/1)
Conservative Majority                     6.8 (6/1)
Any Coalition Involving SNP          8.9 (8/1)
Labour/Lib Dem Coalition             9.1 (8/1)
Labour/SNP Coalition                   11.9 (11/1)
Cons/LD/DUP Coalition                16.3 (15/1)
Any Coalition Involving a NI party  19 (18/1)
Any Other Government/Coalition  13.2 (12/1)
Labour/Lib Dem/SNP Coalition    21.8 (21/1)
Conservative/Labour Coalition     29.4 (28/1)
Any Coalition Involving UKIP         28.9 (28/1)

Conclusions

In terms of betting odds, at the start of the year Labour and the Conservatives were neck-and-neck, but the past few months has seen the Conservatives over take Labour in terms of the odds for a majority of the seats, where the Conservatives sit at 1.2 (1/5) and Labour at 4.4 (10/3). While the bookies think that the Conservatives will gain the majority of the seats, they are almost definite that they predict there will be no overall majority, with a whopping 10-to-1 on (1/10).

The previous analysis around the Scottish Referendum showcased the bookies predicted the result earlier and more precise than the pollsters, so let’s see how this one goes. Just now, for the next Prime Minister, it’s two horse race, with Ed slightly in front.

The amount of changes in odds around the number of seats that UKIP will gain is an interesting observation, especially in terms of spread betting.

So in the end, the bookies are predicting a Labour minority government, but not at a strong price (17/10), and this is followed by a Conservative/Lib Dem Coalition (10/3). Both these odds show that it is still too early to call the final shape of the government.

With OS X Yosemit – Who’s next in Apple’s sights?

Introduction

Apple are well known for their focus on constraining both hardware and software vendors to work within their specifications for Mac OS X and iOS. They are also a company who have the resources to target a successful business area for others, and have the resources to follow through and gain significant adoption. From a company who were floundering with computer systems which did not fit-in with the main-stream, and build with custom chips and with their own operating system, they have effectively become the company who set the standards within the computing industry, and in a way that IBM did in the 1970s. So with the release of Yosemite, we’ll see who Apple have in their sights. As other computer companies sit back, and try not to rock the boat, Apple have shown it has the guts to take on the industry, and steer it in their direction. From since the glory days of IBM has seen such a drive on setting industry standards.

Adobe stand-off

After the computing industry shook off its IBM dominance, Apple’s main competitor was Microsoft, and, through meetings with Steve Jobs and Bill Gates, the two companies managed to work together, especially around the porting of Microsoft Office to the Mac. The main target, though, for Steve Jobs was Adobe Flash, who, in April 2010, wrote an open letter outlining the reason that Apple would not support Flash on the iPhone, iPod touch and iPad. For any software house, the reasons given were a dead-nail, and included:

• Considerable energy consumption.
• Poor security.
• Lack of touch integration.
• Poor performance on mobile devices.

And, to push them off the mobile platform, he dictated that there was a desire to exclude third party software coming between the developer and the operating system, and that Flash, which was claimed to be open software, was a closed system.

This was the beginning of the end for Flash for mobile devices, where developers had to quickly migrate their software towards the iOS, or quickly lose customers. For many Adobe Flash had been the way to develop once for a range of platforms, and possibly this was at the root of the problem for mobile devices, as Adobe were trying to cover so many bases. With this a developer could develop for a desktop, and then the same Flash content would run on both Android and Apple iOS. It was one of the few software plug-ins that could cover all of these, and which seamlessly downloaded the plug-in and associated updates. From an industry point-of-view, the future looked rosy for a platform that could cover the migration from desktops to mobile devices, but Apple’s decision effectively stopped them in their tracks. Apple tried to justify its approach too by stating that online video no longer required Flash, leaving Apple plug-ins such as QuickTime ready to take on the increasing requirements for video streaming.

For all the arguments around it’s cross-platform nature, Adobe Flash has been weak around its security integration, and has ended-up being the rogue’s gallery for software vulnerabilities, being highlighted with one of the three main threats which expose users to malware: CVE-2013-5331 (Adobe Flash), CVE-02007-0071 (Adobe PDF) and CVE-2013-1723 (Java). If a user has an unpatched system, they can be exposed to each of these vulnerabilities.

The threats are fairly easy to implement for script kiddies using exploit kits such as the Phoenix Exploit Kit v2.5, which has all the scripts required to create the documents and the code required to exploit the user’s machine (Figure 1). There’s a whole industry in exploit kits, where, for a maintenance fee, the Exploit Kit creators will patch their exploits to make use of the most up-to-date vulnerabilities, and try and overcome some of the patches applied by venders.

Apple have, though, removed the restrictions related to third-party tools, languages and frameworks, which does allow Flash deployments on iOS using Adobe’s iOS Packager. This approach is still risky for many developers, as the door may be closed on future updates. For Adobe, the have moved their efforts away from a Flash plug-in for mobile browsers, and to port their ActionScript programming language and Adobe AIR towards native mobile apps.

Figure 1: Adobe and Java exploits

Yosemite and the user experience

The clash with Adobe Flash highlights that Apple identified that there was a great worry that customers would move away from the App Store, as equivalent Flash applications would have provided a suitable alternative, especially as applications could be developed to support not only iOS, but also Windows, Linux, and Android. Along with this, Apple are keen to control the user experience, as the new Yosemite, where they have seamlessly integrates a new look-and-feel for the user. As shown Figure 2, Apple have gone for a flatter user experience, and it is one which users take to almost instantly, and actually forget the updates to the GUI. This approach is to not radically change, as we have seen with Windows 8, but to seamlessly update and refresh.
Figure 2: Yosemite OS X

A demo is shown here:

Personal opinion

I have personally spent a great deal of time developing with Flash, but it became along unmanageable to maintain the ActionScript code. So when ActionScript 2.0 came along, I had the decision to port all my code to ActionScript 2.0 or to go with Java or .NET. My choice was .NET, and I’ve never regretted it. Unfortunately .NET is not really that well supported on iOS from a user interface point-of-view, but I’ve generally spent my time developing for the Web, and with ASP.NET MVC, I have the perfect development environment, where I can share code across my applications, and clearly separate the user interface from the middleware. So, here’s one of my old Flash apps:

http://billatnapier.com/hang_ccna.html

The next target?

With Apple now content in constraining the applications and user experience that they gain on OS X and iOS, their next target is Cloud synchronisation, as users are increasingly storing their content onto Cloud-based systems. With a generalised Cloud synchronisation package, such as Dropbox, users can easily update all of their files on a range of device, of which could be both Apple-based or not. This provides users the ability to move and edit their documents on Windows or Mac desktops, and then synchronise on their mobile devices (typically either an iPad or an Android tablet).

The Cloud sync package thus holds a great deal of power in the way that systems are architected, and users will develop trust in their Cloud sync provider, especially as they do not have to backup their system with complex software. And so the target for Apple is Cloud sync apps, and one of the key features of Yosemite is its seamless integration with the iCloud, where files can be synched between Apple iOS and Mac OS X. This is especially focused on users with an Apple iPhone, an Apple iPhone, and a Mac book. For them, from Apple’s point-of-view, the three devices are integrated together, but it risks a closed Cloud environment, which pushes out Windows and Android, and could lead to a lock-in for users.

Conclusions

Adobe possibly tried to cover too many bases with their Flash plug-in, and forgot that mobile devices were different beasts to desktops, and where the consumption of energy and CPU were not good things. They slipped too in consider security considerations for the desktops, and have become part of the rogues gallery, being the exploit of choice for malware writers. The writing was on the wall, though, with the move towards HTML 5, and the track for Flash was always going to be a difficult one, especially as Flash has always been seen as a resource hogger.

With the latest integration of Yosemite OS X, we can see the coming together of a range of devices to seamless integrate them. For Apple, they would like users to have all three of their devices, so Yosemite sees the first real attempt to bring them together using the iCloud. Thus Dropbox and Cloud sync providers are now a target for Apple, and, for a company that Steve Jobs tried to buy many years ago for its iCloud, it has managed to develop a strong business, but one will see if it can stand up against the might of Apple. Some small integration issues around the installation of Yosemite perhaps show it will be a bumpy ride for Dropbox.

Analysis of UK General Election Betting – 2 May 2015

The latest analysis is here.

Introduction

Previously I have analysed the betting patterns around the Scottish Independence referendum, and found some significant trends (here). So, while the UK election is still a long way of, this blog will analyse some of the trends within the UK election, and see if we can use some of the pointers from the Scottish Independence referendum to highlight possible signposts. One thing that was highlighted in the analysis of the referendum is that the bookies actually predicted the result, and give pointers to the geographical votes, at an earlier stage than many of the analysts.

Outline of odds

In the UK election poll, there are four horses in the race, so over the next few months we’ll be analysing the key turning points in the odds. The way that odds are normally defined is the fraction which defines the return, so Evens is 1/1, where for every £1 bet, you will get £1 back in addition to your stake (so you get £2). If the odds are 2/1 (2-to-1 against), you get £2 back plus your stake (so will get £3 on a win). For 1/2 (or 2-to-1 on), you get half your money back, and you’ll get £1.50 on a win. These types of odds are known as fractional odds, where the value defines the fraction for your payback. The multiplier, though, does not show your stake coming back to you, so decimal odds are used to represent this, and defines a value which is multiplied to the stake to give the winning amount (basically just the fractional odds plus 1, and then represented as a decimal value).

The factional odds value of Evens gives a decimal odds value of 2 (where you get £2 back for a £1 stake), and 2/1 (2-to-1 against) gives 3.0, while 1/2 (2-to-1 on) is 1.5. In terms of roulette, Evens would define the odds for a bet of Red against Black (as each are equally probable). In roulette, though, the odds are slightly biased against the player for a Red v Black bet, as 0 changes the odds in favour of the casino. For betting, overall, bookmakers try to analyse the correct odds so that they have attractive ones (if they want to take the best), against others. If they take too much of a risk, they will lose, so their odds around the election vote should be fairly representatives of the demand around bets, and the current sentiment around the debate.

Current betting

As of 2 May 2015, the average decimal bets are given in Table 1. It can be seen that the Conservative are 5-1 on, in winning most seats, with Labour sitting at 10/3.

Table 1: Current average odds

Labour	4.3 (approx 10/3)
Conservative	1.2 (approx 1/5)
UKIP	413 (412/1)
Liberal	929 (928/1)

Majority or not?

In terms of a majority or not, the bookies are predicting that there is not likely to an overall majority, and this sits at 1/10, which is the kind of odds you would struggle to get even if Barcelona where playing Luton Town, at home. The Conservative party are the closed to an overall majority, but still sit out at 6/1, with Labour at 40/1.

No Overall Majority	1.1 (1/10)
Conservative Majority	7 (6/1)
Labour Majority	41 (40/1)
UKIP Majority	241 (240/1)
Any Other Party Majority	583 (582/1)
Green Majority	1001 (1000/1)
Liberal Democrat Majority	819.2 (818/1)

Trends

From a starting point of being equal for betting around Sept 2014, the Labour party has generally drifted out from 1.8 (4/5) at the start of Jan 2015 to 4.4 (10/3) on 2 May 2015 (Figure 1), while the Conservatives have moved from 2 at the start of Jan 2015 to 1.2 on 2 May 2015 (Figure 2).

Figure 1: Analysis of the Labour’s odds for winning most seats (6 Aug to 27 Oct 2015)

Figure 2: Analysis of the Conservative’s odds for winning most seats Dec 2014 to 2 May 2015)

Predicted number of seats

While the Conservatives are favourites for the majority of the seats, the favourite for the number of seats they will gain has not varied much of the past few months (Figure 3), with a current estimation at 286.5 seats as apposed to 282.8 in Jan 2015. Labour, though, have slipped for the number of seat from 287.5 to 268.8 (Figure 4).

Figure 3: Predicted number of seats for The Conservatives (Jan 2015-May 2015)

Figure 4: Predicted number of seats for Labour (Jan 2015-May 2015)

In terms of UKIP, the predicted number of seats from the bookies around the start of the year was 6.5, and this has fallen to 2.6 average prediction (Figure 5).

Figure 5: Predicted number of seats for UKIP (Jan 2015-2 May 2015)

For Liberal Democrats the trend for seats from betting has not shown much change with a variation from a starting point of 27.5 to a current threshold average of 28.8 seats (Figure 6).

Figure 5: Predicted number of seats for UKIP (Jan 2015-2 May 2015)

So the current standing from the bookies for the average threshold for seats is:

Conservatives        282.8
Labour                     268.8
SNP                         51.5
Liberal Democrats 28.8
Plaid Cymru            3.5
UKIP                        2.6
Greens                    0.5

The Next Prime Minister?

In terms of the average betting on 2 May 2015, the next prime minister it is very much looking like it is level-pegging with Ed and David, with Ed Miliband slight more of a favourite at 4/5 and David Cameron on Evens:

Ed Miliband             1.8 (4/5)
David Cameron      2 (Evens)
Caroline Spelman   51 (50/1)
Boris Johnson         61.9 (61/1)
Tristram Hunt           67 (66/1)

Scottish Analysis

In Scotland, the bookies are currently pitching +51.5 seats as a key breakpoint for SNP seats. If we look at the trend, the bookies have been moving up the breakpoint bet for the number of seats they will gain from +6.5 in April 2014 to the current value of +51.5, and have increased by 43 seats (Figure 6).

30 Apr 2014: +6.5
26 Aug 2014: +7.5
18 Sept 2014: Referendum vote
22 Sept 2014: +8.5
10 Oct 2014: +10.5
19 Oct: +11.5
27 Oct: +12.5
2 May: +51.5

Figure 6: Predicted number of seats for SNP  (Sept 2014- 2 May 2015)

The bookies are thus predicting that the SNP will gain 51.5 out of 59 seats available. Only one bookie is laying bets on them winning all the seats, and it is pitched at 6 (5/1).

Most activity on betting?

Punters will generally be looking for a good place for a bet, and aim to get in early before any movements to bring the odds in. From the start of the year, the number of bet odd changes have been:

UKIP 1,197 bet changes
SNP 212 bet changes
Liberal Demoncrats 50 bet changes
Liberal 48 bet changes
Labour 47 bet changes
Conservatives 30 bet changes

It can be see that perhaps the number of seats that UKIP will gain is the most variable factor in the betting dynamics.

Conclusions

In terms of betting odds, at the start of the year Labour and the Conservatives were neck-and-neck, but the past few months has seen the Conservatives over take Labour in terms of the odds for a majority of the seats, where the Conservatives sit at 1.2 (1/5) and Labour at 4.4 (10/3). While the bookies think that the Conservatives will gain the majority of the seats, they are almost definite that they predict there will be no overall majority, with a whopping 10-to-1 on (1/10).

The previous analysis around the Scottish Referendum showcased the bookies predicted the result earlier and more precise than the pollsters, so let’s see how this one goes. Just now, for the next Prime Minister, it’s two horse race, with Ed slightly in front.

The amount of changes in odds around the number of seats that UKIP will gain is an interesting observation, especially in terms of spread betting.

Lollipops and Apples – Entering The Crypto Age!

Introduction

Google’s Lollipop will be released next week, and security will be at the core of its changes. An important element of this is in encryption-by-default, where users will have to opt-out of encryption of their files. Apple, too, with iOS 8 have taken the same route, and users must ask: “Why didn’t it happen before this?”

Our file attributes and content types have developed with little thought on keeping things truly private, and where systems are often still viewed as stand-alone machines. We also created an Internet which is full of the same protocols that we used in the days of text terminals and mainframe computers, where users typed in commands to access data, and where there was little thought about protecting the data as it is stored, analysed and transmitted. As we are increasingly move mobile, we are now carrying around our sensitive data, that at one time was protected behind physical firewalls, and the risks to our data increases by the day.

The major tension, though, is between law enforcement and the right to privacy. The FBI currently see the status quo as a way of investigating criminals and terrorists, but can see this opportunity reducing with encryption-by-default, such as with the file encryption system used in Apple’s iOS 8. With iOS 8 and Google Lollipop there will be no electronic methods to access encryption keys from existing digital forensics toolkits, and thus the encryption method breaches current laws, which force users to reveal their encryption keys when requested by law enforcement investigators. This would mean that users may be breaching current laws in both the US and the UK. The same battle too exists with Tor, where law enforcement are scared that crime can go un-noticed, whereas privacy advocates promote the rights of privacy of using Tor.

No right to remain silent with Cryptography

In the UK, citizens have the right to silence (a Fifth Amendment Right in the US – related to the right against self-incrimination) but there is an exception to this related to encryption keys, and the failure to reveal encryption keys can often be seen as a sign that someone has something to hide, and is covered by Section 49 of RIPA. The move by Apple and Google may thus breach law as they must be able to hand-over their encryption key when required. This was highlighted in 2014 when Christopher Wilson, from Tyne and Wear was jailed when he refused to hand encrypted passwords related to investigations related to an attack on the Northumbria Police and the Serious Organised Crime Agency’s websites. He handed over 50 encrypted passwords, but none of these worked, so a judge ordered him to provide the correct one, but after failing to do this, he received a jail sentence of six months.

In 2012, Syed Hussain and three other men, were jailed for discussing an attack on a TA headquarters using a home-made bomb mounted on a remotely controlled toy car. Syed, who admitted have terrorist sympathises, was jailed for an additional four months for failing to hand-over a password for a USB stick.

The following outlines some key features in disk encryption:

The Perfect Storm

The main problem that we have with computer system security is that as computer systems have evolved we created file systems which only protect using file attributes. This works well from a corporate point of view, where we can keep compatability with previous systems, and also allow system administrators to keep full control of them. The mobile device operating system creators (mainly Google and Apple), though, have different issues to the traditional desktop operating system creators, as their devices are on-the-move, and often stolen or left behind.

As we increasingly integrate the mobile phone with our lives, especially in creating a digital shadow on the Cloud, the devices need to be more protected that our traditional desktops. Along with this, Apple and Google have complete control over their operating systems, and can implement radical changes in a way that Microsoft would have struggled with (and still keeping compatibility with an operating system released over a decade ago: Windows XP). So Apple and Google are not constrained by the past, and find their hardware platforms are whizzing along with increased processing speeds and memory capacities, in a way, again, that Microsoft would struggle with, as they have so much legacy hardware that would struggle with modern cryptography.

So Apple and Google now find themselves with a market that will quick change their mobile devices and keep up-to-date, and this do not have the long tail of devices to support. If a user wants to stick with a certain operating system, they can, but there’s a good chance that their applications won’t work. With phone manufacturers pushing new phones all the time, both Apple and Google are keen too to plug the gaps in traditional operating systems, especially related to security, and they have the perfect storm with SSD (rather than the horribly slow HDDs), and fast multi-core processors, each which now make encryption possible on a device that fits in your hand. Gone are the days when you needed a special maths chip to do complex cryptography.

Some basics

It is important to understand how disk encryption typically works, as weaknesses can be identified. Overall there is no way method that is the best for securing a system, and it basically comes down to the risk level on the data contained on it. As Figure 1 illustrates, the four main methods are: to have a passphase; store key(s) on usb drive; generate access code from a OTP (One-Time Password); or use a biometric device. These methods typically allow access to the encryption keys which are used to secure the encrypted files. On many systems the encryption keys are held on a digital certificate on the host (or on the domain controller), but these can often be opened using password cracking on the certificate. Along with this, if we encrypt the whole disk, it will be difficult to get access to the digital certificate on the host, as it is part of the encrypted system. Microsoft Bitlocker gets round this problem by having two disk partition, and where one can hold the protected encryption keys.

Figure 1: Disk encryption access methods

Increasing, though, the method of using a digital certificate is difficult to sustain, and thus the move is towards a TPM (Trusted Platform Module) which embeds the encryption keys into a chip on the device. The operating system boot process then is able to access the encryption keys, and where they are protected by one of the methods defined in Figure 1. For both Apple and Google, TPM is at the core of their approach for encryption-by-default, and where the user has control over the release and methods of security around the encryption keys. If they use a PIN number, their keys are easily found, but pass phrases make it much more difficult.

Figure 2: TPM

Public and Private Keys working in harmony

Many users, even computer science graduates, thing that we either have public or private key for our secure systems, but often the two work in harmony, and focus on what they are good at. With private key, such as AES, we have a high optimized encryption method, which is fast, especially if we have the right key. Thus most encryption that happens on the disk is private key (typically 256-bit or 128-bit AES). The protection of the key then is done by public key, and where a public key encrypts the File Encryption Key (FEK), and where it can only be decrypted by the private key (Figure 3). In this way, public key does the protection of the key, and private key is the workhorse of the reading and writing of the data. The other method that is used is to generate a key based on a passphrase, and where we add some salt to it, to make sure it is an ever changing key. Overall, though, public key encryption, such as RSA, is hardly ever used in disk encryption, as it is such as slow method. As RSA keys move toward 4,096 bits, it is become increasingly difficult to process large amount of data, in real time.

Figure 3: Protecting the FEK (File Encryption Key) with public key

From File Attributes to File Encryption

At present files are typically secured by file attributes, which are acceptable on desktop systems, especially especially ones which connect to domains, but on mobile devices it is extremely difficult to define protection levels. For Unix-type systems we have simple attributes of:

drwxr-xr-x   16 billbuchanan  staff   544B 28 Sep 19:27 mydir
-rw-r--r--     1 billbuchanan  staff   201B 16 Apr  2014 results.txt
-rw-------     1 billbuchanan  staff   210B 16 Apr  2014 run.save
-rw-r--r--     1 billbuchanan  staff   194K 11 Jul 19:04 salt.svg

where were we have r(ead), w(rite) and e(x)ecute for the owner, their group and the rest of the World. In terms of keeping things simple, this is about as good as it gets, but it is often difficult to define other rights, such as for deleting and creating a file. So NTFS defines other attributes of F (Full Access), M (Modify Access), D (Delete Access):

 perm is a permission mask and can be specified in one of two forms:
     a sequence of simple rights:
             N - no access
             F - full access
             M - modify access
             RX - read and execute access
             R - read-only access
             W - write-only access
             D - delete access  

C:\dropbox>icacls *.zip
networksims_accounts.zip NT AUTHORITY\Authenticated Users:(I)(F)
                         NT AUTHORITY\SYSTEM:(I)(F)
                         BUILTIN\Administrators:(I)(F)
                         BUILTIN\Users:(I)(F)

tables_xp_free_fast.zip NT AUTHORITY\Authenticated Users:(I)(F)
                        NT AUTHORITY\SYSTEM:(I)(F)
                        BUILTIN\Administrators:(I)(F)
                        BUILTIN\Users:(I)(F)

Successfully processed 2 files; Failed processing 0 files

And then lots of extensions on the inherited rights:

 (OI) - object inherit
 (CI) - container inherit
 (IO) - inherit only
 (NP) - don't propagate inherit
 (I) - permission inherited from parent container

So NTFS works extremely well in managing the access and rights to files on a domain, and where a domain controller defines the rights for the files at a central point. It only defines the overall “owner” as the system administrator for the complete domain, so as long as it connects to the domain, the administrator has complete rights to files. This type of approach can thus be used in any investigation, where the rights on the files can be changed to suit the investigation. On a mobile device, it would be difficult to implement such complex rights, especially as many of the systems are Linux-based, so encryption is the natural way to protect files, and where the user themselves have control over their access.

Conclusions

With data breaches rising by the day, such as with 150 million passwords cracked with the Adobe infrastructure and over 120 million credit card details skimmed for Home Depot and Target, Apple and Google feel they have to build up trust with their users in their operating system. For this they are looking at encryption-by-default, where they encrypt file data (which is now stored on flash memory), and which now may breach the laws around reveal encryption keys. At one time, investigators could extract the memory from the device, and decode its contents, but without encryption keys this will be difficult. While Google and Apple have not responded to the dilemma, there could be the opportunities for them to work with the companies to overcome of the issues, which might reduce privacy settings on their data. Unfortunately if they do reduce the security on the encrypted data, they may leave open opportunities for others to learn the methods, and compromise the whole system. In a corporate market, Microsoft BitLocker is one of the most popular methods used for complete disk encryption. With this, there is always the back-door input into the encrypted data, by storing the encryption keys within the domain controller for the company.

For our rights systems, we are moving away from complex file attributes to protect files, toward simpler methods which define that we encrypt all content by default. Our old desktops have held the industry back for so long – with their lumbering magnetic hard disks and their separation of disk storage and memory storage. For mobile devices they have electronic memory for both running programs and storage, and will typically run 100s of times faster than their mechanical and magnetic brother. So for them, encryption by default is a natural extension, and with modern crypto methods such as AES, we have finally entered a new era of computing – The Crypto Age!

We have a long tail left of legacy with computer systems, typically through slow disk systems, limited processors, and a lack of memory. This shackles are now off for mobile devices, and they are free to push forward and properly integrate security, which must be build on a core of cryptography.

For the rights system on a mobile device we might have “It’s mine!”, and that’s it. From a corporate point-of-view, this will not be acceptable, so many system developers are working on properly integrating devices into the core of the infrastructure, and encryption by-default should aid this process, and not hinder it.

From a Murmor to a POODLE

Introduction

Few things can be kept secret within computer security, and it has been the case with a flaw in SSLv3, where there were rumours of a forthcoming announcement. So it happened on Tuesday 14 October 2014 that Bodo Möller (along with Thai Duong and Krzysztof Kotowicz) from Google announced a vulnerability in SSLv3, and where the plaintext of the encrypted content could be revealed by an intruder. The flaw itself has been speculated on for a while, and this latest announcement shows that it can actually be used to compromise secure communications.

The vulnerability was discovered last month, and named POODLE (Padding Oracle On Downgraded Legacy Encryption) attack, and it relates to the method that Web servers deal with older versions of the SSL (Secure Socket Layer) protocol.

Secure Sockets and Tunnelling

Figure 1 outlines how network protocols fit together, where we use insecure protocols such as HTTP, TCP and IP to communicate with a Web site. Each of these protocols were designed at a time when security was not an issue on systems. The fix to improve security was to insert a layer between the network layer (IP) and the transport layer (TCP), and is named SSL. This later secures the upper level protocol, and creates a secure and encrypted tunnel between the client and the Web server, so that no-one can crack the communications for the connection.
Figure 1: Networking stack with SSL integration

There are several different versions of SSL, from 1.0 to 3.0 , and with 3.1 it changed its named to TLS (Transport Layer Socket), so that SSL 3.1 is also known at TLS 1.0. Most systems now use TLS 1.1 or 1.2, and which are free of flaws which compromised previous versions.

SSL (as illustrated in Figure 2) works by the client telling the server which type of encryption it would like to use (such as RC4) and the other cipher parameters it can support. The server replies back with its preferred cipher scheme, along with its digital certificate. This digital certificate contains its public key, and the proof of its identity. The client then creates a new session key for the encrypted tunnel, and then encrypts this with the public key provided by the server. It then sends this to the server, where the server decrypts it, and thus has the same encryption key as the client. After this, the client and server can communicate with the session key (and the chosen encryption method) – thus creating a tunnel between themselves. This tunnel should be almost impossible to crack, as it uses a session key, which would take a long time to crack through brute force.Figure 2: Outline of SSL tunnelling

SSL flaw

Within SSL, most servers will cope with previous versions of the protocol, and will downgrade from TLS 1.0 to SSLv3 if the handshake between the client and the server fails. The intruder will thus refuse a TLS 1.0 version, and go for an SSLv3 method, where they will be able to decipher the communciations. In the following example, we force the connection to the Google Web site to use an SSLv3 connection (with the -ssl3 option):

billbuchanan@Bills-MacBook-Pro:~$ openssl s_client -connect www.google.com:443 -ssl3
 CONNECTED(00000003)
 depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA
 verify error:num=20:unable to get local issuer certificate
 verify return:0
 ---
 Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
 i:/C=US/O=Google Inc/CN=Google Internet Authority G2
 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2
 i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
 i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
 ---
 Server certificate
 -----BEGIN CERTIFICATE-----
 MIIEdjCCA16gAwIBAgIISVyALWN+akUwDQYJKoZIhvcNAQEFBQAwSTELMAkGA1UE
 BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl
 --- missed out content
 fsX5GyPM24FrA9G3tbBOrDBrclbG8tBhSS+yIS2e4D3WpVrqiYDr9YqOmpD8jXWH
 SOx4I5L0D0jZYqKfJuImGcFwdIETq0EpCmkhJfGNHjVdzC/h/T61TmaY
 -----END CERTIFICATE-----
 subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
 issuer=/C=US/O=Google Inc/CN=Google Internet Authority G2
 ---
 No client certificate CA names sent
 ---
 SSL handshake has read 3578 bytes and written 299 bytes
 ---
 New, TLSv1/SSLv3, Cipher is ECDHE-RSA-RC4-SHA
 Server public key is 2048 bit
 Secure Renegotiation IS supported
 Compression: NONE
 Expansion: NONE
 SSL-Session:
 Protocol  : SSLv3
 Cipher    : ECDHE-RSA-RC4-SHA
 Session-ID: F220B72DE15D22EB0AE909DBF25C1731FEE98B4D77E5AB123A6648425DADA398
 Session-ID-ctx:
 Master-Key: B3CB1EA901EEFCA6A2017E3C3E7DDBB5037FA171D20886A6C25C481D008F23535D2345E7E274704C1398ED138D05C6BD
 Key-Arg   : None
 PSK identity: None
 PSK identity hint: None
 SRP username: None
 Start Time: 1413353089
 Timeout   : 7200 (sec)
 Verify return code: 20 (unable to get local issuer certificate)

The problem around SSLv3 has been around for a while, and many in the industry professionals have been recommending that administrators do not allow its usage.

Conclusions

The flaw is not a Heartbleed-type issue, which caused major compromises across the Internet. Overall it is basically highlighting a previously defined flaw. A more pressing issue with SSL/TLS is Man-in-the-Middle attacks, and where an intruder can get in-between the client and the server (using a proxy connection). The fix for POODLE is simply for administrators to disable SSLv3 on their Web sites, in order to avoid the downgrade of the secure connection. Even still, it’s not a major problem, as the intruder is just reading their own data.

A video lecture on SSL/TLS will appear here later today.

Dropbox Accounts Exposed? Multi-factor Authentication and Encryption By Default

Introduction

The latest comprise of Dropbox accounts has nothing to do with a security breach at Dropbox, and relates to the hacks of other systems, but where users use the same email address and passwords across a range of systems. This type of hack is a particular problems in computer security, as users often have just a few passwords, and often use the same password for all of their logins. The originators of the hack claim that they have seven million compromised passwords, and where asking for Bitcoin donations for the release of the information, but, as of yesterday, had only managed to raise 4 cents worth of funding. It is reckoned that out of the 220 million users of Dropbox, that only around 3% of these accounts (6,937,081) could be at risk from the latest release of passwords.

A similar thing happened to Dropbox in 2012, where usernames and passwords were stolen from other sites and compromised a number of Dropbox accounts. For this, users found a folder such as:

16-Feb-12  03:15 PM    <DIR>          Your DropBox has been Hacked

which identified that a hacker had gained access to their Dropbox account. In 2011, too, Dropbox inadvertently published code on its own website allowing anyone to sign into anyone’s Dropbox account without any login credentials.  These issues have caused many to move towards multi-factor authentication for Dropbox storage.

Multi-factor authentication

For most, there is only one way to use Dropbox, and that is to use multi-factor authentication. In this way, just gaining access to someone’s email address and password will not allow access to their Dropbox account. In any multi-factor authentication, users are challenged with two or more of: “Something they know” [such as a username/password], “Something they have” [such as a smart card], “Something they are” [such as a fingerprint] and “Somewhere you are” [such as your location]. Increasingly systems are being designed to integrate these challenges, as usernames and passwords are becoming difficult for users to remember, and even when used correctly, often it is password for hackers to reverse the stored hashed version of the password (as have been seen with the Adobe hack, which involved the compromise of over 150 million usernames and passwords). Often we use out-of-band methods to allow the user to identify themselves, such as sending an SMS message through a mobile phone, which is then entered into a Web page for access. This is often more secure, as it is difficult for an intruder to gain access to both username/password and mobile phone device.

Figure 1: Multi-factor authentication

Encrypting into the Cloud

Increasing users are using Cloud-based systems, such as with Dropbox and SkyDrive, in order to share files with other users. With potential drive sizes which are greater than the disk space that corporations can offer, it is now an excellent method to store and share files. One of the best solutions is TrueCrypt, which is free and allows users to store files to an encrypted file contains. For more information on disk encryption:

HBGary Federal

The HBGary Federal example is the best one in terms of how organisations need to understand their threat landscape, and where a single password was used for a range of accounts. It started when Aaron Barr, the CEO of HBGary, announced he would unmask some of the key people involved in Anonymous, and contacted a host of agencies, including the NSA and Interpol. Anonymous bounced a message back saying that they shouldn’t do this, as they would go after them. As HBGary were a leading security organisation, they thought they could cope with this and went ahead with their threat.

Anonymous then searched around on the HBGary CMS system, and found that a simple PHP request of:

http://www.hbgaryfederal.com/pages.php?pageNav=2&page=27

gave them access to the complete database of usernames and hashed passwords for their site. As the passwords were not salted, it was an easy task to reverse engineer the hashes back to the original password. Their target, though, was Aaron Barr and Ted Vera (COO), each of which used weak passwords of six characters and two numbers, which are easily broken.

Now they had their login details, Anonymous moved onto other targets. Surely they wouldn’t have used the same password for their other accounts? But when they tried, the can get access to a while range of their accounts using the same password (including Twitter and Gmail). This allowed Anonymous access to GBs of R&D information. Then the noticed that the System Administrator for their Gmail Email account as Aaron, and managed to gain access to their complete email system, and which included the email system for the Dutch Police.

Figure 2: Access to email and a whole lot more.

Finally they went after their top security expert: Greg Hoglund, who owned HBGary. For this they send him an email, from within the Gmail account, from a system administrator, and asking for confirmation on a key system password, of which Greg replied back with it. Anonymous then went onto compromise his accounts, and which is a lesson for many organisations. While HBGary Federal has since been closed down, due to the adverse publicity around the hack, the partner company (HBGary) has went from strength-to-strength, with Greg making visionary presentations on computer security around the World.Figure 3: Greg’s compromise

Conclusions

For many who have seen the problems around public cloud-based storage, the only solution is multi-factor authentication, and the mobile phone is typically the device of choice in properly identifying the individual (especially through out-of-band authentication). Along with this users need to also look towards encrypting their data into the Cloud, as no remote storage can be completely free from malicious accesses. TrueCrypt, for example, allows users to encrypt data into the Cloud, so even if someone gets access to the cloud data, they will not be able to read it (as it requires a secure password). So, for companies, the only way forward in using public cloud storage, is to both implement multi-factor authentication, and, if possible, to encrypt into the public cloud.

Users are being advised to move toward multi-factor authentication, but that is only the first step in moving towards encryption-by-default.

What? Our ATMs are still running Microsoft Windows?

Introduction

Over the past decade we have seen some major viruses and worms infect systems running Microsoft Windows, where computers were often infected by a Trojan which then installed a malicious program, which then sustains itself by updating the registry file for the auto-startup on the system. In this way the malicious program would stay resident on the computer, and would often change its name to defeat defence systems. The malware would often only start resident for a short time on the host, and even try to disable the security controls to avoid being detected. On a more secure and locked-down system, it was more difficult for the malware to be downloaded or installed on the system, so the way round this was to force the Windows host to boot from a bootable CD, which give administrator access to the machine, and then could be used to install malicious software, with the normal controls that the operating system would place on the system. This is basically what has happened with the recent ATM back, where operators booted the ATM system from CD, in order to install the malware, which was then access at set times of the week, and with a randomly generated six-digital code. It seems amazing in these days or more secure systems, that a simple boot from CD on an ATM can result in money being withdraw at will. Figure 1 provides an overview for the hack.

Figure 1: ATM hack

Malware on Cash Machines

There are two key principles in computer security:

• The overall security of the system is only as strong as the weakest link.
• Systems need to be designed to reduce the surface area for an attack.

Unfortunately, as the back-end systems become more secure, intruders are focusing on the development of malware at the front-end device, and where the surface area for attacks is at its greatest. Recently Home Depot , in the US, discovered that at least 56 million credit and debit card details have been compromised from all of it 2,200 stores in the United States, using a malware agent running on the PoS (Point of Sale) devices, and which has been running from April 2014 to the beginning of September 2014 before it was detected. This came after a similar hack at Target hack which exposed an estimated 40 million cards, and where a large number of credit and debit cards have appeared on the credit card clearing house site: rescator.cc .

Skimming devices are well known within ATMs, where an integrated camera and a card skimmer are used to read the details of the pin number and the card details when the user tries to withdraw cash. A while back the manufactures selected their operating system of choice, and in many cases it was initially OS/2 (which was developed jointly by IBM and Microsoft), but it is increasingly a standard build of Microsoft Windows. As Windows is often open to a range of vulnerabilities, it is now the case for ATM machines.

With the use of standard operating systems, such as Microsoft Windows, it has not taken long before intruders could probe the operation the card processing system in the ATM. Early this year Bruce Schneier outlined a new method of encoding malware onto the bank card and drops a file onto the ATM (isadmin.exe file). The isadmin.exe file was then used to replace lsass.exe (which has previously been compromised by the Sasser Worm) with a malicious version, and which then collected credit card details and PIN numbers. These details are then harvested when the hacker inserts a special control card into the ATM to gather user details, and, possibly, ejecting the cash storage unit.

The latest malware is Backdoor.MSIL.Tyupkin, and which is continually running on the system, but will only accept input commands on a Sunday and Monday night, uses a six digital key sequence that can only be generated by the gang leader. When entered correctly, the ATM displays the amount of money in the cassettes, and allows up to 40 notes to be extracted from the ATM.The malware differs from the one outlined by Bruce Schneier as it requires physical access to the ATM to install the malware, where the operator inserts a bootable CD into the ATM controller and which installs the malware. The money gatherer then gains access the ATM with a special six digit code, which is only told to them when they are ready to withdraw the money, as the malware gang do not want anyone going it alone. At present the malware has been active on at least 50 ATMs in Eastern Europe, but could also be infecting units in the USA and India.

Technical details

The installed malware basically copies the malware file of ulssm.exe into the c:\windows\system32 folder and which is sustained on the system by modifying the [HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run] registry key (Figure 2). This registry key is used to run programs automatically on startup, and thus the malware will stay on the ATM, when it is rebooted.  Once infected it then interacts with ATM through MSXFS.dll (Extension for Financial Services – XFS), and to avoid detection it will only allow access controller commands on Sunday and Monday evenings.

Figure 2: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run registry key example

The following shows an example of malware installing itself onto a system, and then hiding, and updating the Windows registry to run itself when started (go to 25 mins 20 seconds for the examination on the registry for the Run registry key):

Conclusions

The current threat of CD booting and malware installation on ATMs, with sustained access through a start-up registry key, has been on standard desktops for the last decade, and few lessons have been learnt in terms of the security for physical access to the devices, and also in the rights that malware software can gain on the system. While many companies will focus on the interface with the user, it is often the debugging and diagnostic side which can provide ways into a system.

Most embedded control systems now too are locked-down so that no additional system can be installed, but the choice of standard builds of Microsoft Windows seems to provide easy mechanisms for malicious updates. Microsoft Windows, too, as the core operating systems for ATMs seems to be putting ATMs at great risk, especially it is allows hackers the opportunity to simulate and craft their malware on well-known versions of the operating system.

At the core of this attack is the physical access to the device, and thus access needs to be carefully monitored, but the key lesson, is that the operating system needs to be completely locked-down, and which provides only the software components required to accept user input and dispense of the cash in a reliable way. To still rely on Windows Registry keys, and for them to be updated by the booting of a CD, and to allow malicous programs, seems archaic and where we were over a decade ago with our desktops.

Goodbye Windows security problems – Hello to Mac vulnerabilities

Introduction

After years of Microsoft Windows operating system vulnerabilities, we find that the new place which has moved to discovering sloppy programming in Java, Abode Flash, and Adobe Reader, and now Linux – with a common denominator typically focused around the C++ programming language. The issues around the Linux environment, causes Apple users problems, as Mac OS X has built its environment based on an underlying Linux environment.

In the days when Apple struggled to compete against Microsoft and Intel, and where the company nearly crashed, they made two major choices which completely changed their fortunes:

• Move their architecture from Motorola/IBM (which was build on 68xxx processors) to the x86 architecture used by Intel. This was a smart move as it allowed them to keep up-to-date on hardware against the other PC manufacturers. From a software point-of-view, their programs changed from running Motorola codes to Intel ones, and they changed radically from “Big Endian” programs (where data is stored with the most significant byte first in memory location), to “Little Endian”, where the least significant byte is stored first.
• Move their operating system to Linux. This allow them to reduce their development of their core operating system and focus on the user experience, but still use the reliability of the Linux operating system.

Thus the “special one” is basically just a customized Linux workstation (/server) using standard PC hardware, with a nice user interface on top.  Unfortunately the move to Linux might be coming back with anger on Apple, as serious flaws are now being found in the core infrastructure. Many bugs, which have gone unnoticed for many users, are being found, and these are often due to a lack of code review and testing. With OpenSSL, for example, and which caused Heartbleed, we still see C++ code developed Eric Young (“eay” lib) in 1998.

The other major rise at the current time is in pirated apps for the Mac, such as for Microsoft Office and Adobe Photoshop. When installed these apps provide admin access to the system, and can thus add a whole load of extra things and even reconfigure the computer to be part of a botnet.

iWorm

Last week, Dr. Web (a Russian security company) outlined a new worm for OS X: “Mac.BackDoor.iWorm“, and which allows hackers to take control of the computer, and using it as part of a Botnet, such as for sending spam emails or performing Denial of Service attacks against web sites. Apple, on detecting over 17,000 users with the worm then was quick to respond, and added the signature of iWorm to its malware detector (Xprotect – and which is installed on every Mac computer). A key factor for users is that there Mac needs to be up-to-date with system updates, otherwise they can be open to the spread of the worm. The Xprotect signature defined is:

This worm has been used to search Reddit for a fake discussion forum related to Minecraft and also integrated links to hacker controlled command servers. Reddit then closed the hacker accounts that were used in order to share links to the commands servers, and banned the fake Minecraft subreddit – which stopped the worm from receiving orders from the hackers.

Pirate Apps + Admin Privilege = Problems

It is though that the worm itself was spread though pirated software, such as for Microsoft Office, Adobe Photoshop and Parallels (the virtualisation package used to run other operating systems on the Mac) and downloaded from The Pirate Bay. The problem with installing pirated apps on the Mac, is that they run with Admin privilege, which, just like using an administrator account on a Linux system, they can install a range of other software packages, and also have rights to modify the configuation of the system. It is thus not too difficult from the pirated software to setup backdoors on the machine, just by enabling a network port to be open for connection.

While it is named “iWorm” it is technically a trojan, as it infects the system by users being tricked into downloading malicious software from software which looks valid. To find out if you are protected, on your Mac, open up a command line console, and navigate to:

/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources

and there is a file there named XProtect.plist. Use ls -l to look at the date. The update is likely to have a timestamp of 5 October 2014 and the contents should show the signatures, such as:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<array>
    <dict>
        <key>Description</key>
        <string>OSX.iWorm.A</string>
        <key>LaunchServices</key>
        <dict>
            <key>LSItemContentType</key>
            <string>com.apple.application-bundle</string>
        </dict>
        <key>Matches</key>
        <array>
            <dict>
                <key>Identity</key>
                <data>
                wIAM1QlbKNpLbKAUaKJ5+1vmkho=
                </data>
                <key>MatchFile</key>
                <dict>
                    <key>NSURLNameKey</key>
                    <string>Install</string>
                </dict>
                <key>MatchType</key>
                <string>Match</string>
            </dict>
        </array>
    </dict>

Shell Shock

With Shell Shock the target was again Linux servers using Bash (GNU Bourne Again Shell), which is the command line interpreter used in many Linux based systems, including Apple OS X. Bash interprets the commands that users enter or are run from scripts, and then makes calls to the operating system, such as for running programs, listing the contents of a directory, or in deleting files. The discovered flaw allows intruders to remotely run arbitrary code on systems such as Linux servers including for web servers, routers, and many embedded systems. It was discovered by Stephane Chazelas of Akamai, who found that code at the end of a function of an exported variable is run whenever an environment variable is used within the Bash environment. Many Linux programs use environmental variables to pass parameters between programs, and the flaw thus allows for code to be inserted into a program whenever these environmental variables are called.

Shellshock focuses on CGI scripts, which are old-fashioned scripts that allow commands to be processed using a scripted language. While popular in the past, it has been largely replaced by PHP and other high-level scripting programs. In most cases CGI scripts reside in the /cgi-bin folder. For GNU Bash through 4.3, trailing strings after a function are processed in the definitions of environment variables. This allows intruders to execute arbitrary cod. For example, we have a function named mybugtest:

billbuchanan@Bills-MacBook-Pro:/tmp$ export mybugtest='() { :;}; echo I AM BUGGY'
billbuchanan@Bills-MacBook-Pro:/tmp$ bash -c "echo Hello"
 I AM BUGGY
 Hello

Shell Shock can comprise a system by injecting a payload of code into the environment variables of a running process. When the process is started, the code is injected into the running program, in the same way as a user typing in some user input.

The code which can appear at the end of the Bash function can be fairly complex, and allow an intruder to inject code into the shell (and thus into running programs). In this example we copy some text into a text file (named newfile) and then copy the file to a new file (newfile2):

$ export mybugtest='() { :;}; echo "This is my new file..." > newfile; cp newfile newfile2'
$ bash -c ""
Segmentation fault: 11
$ ls myf*
myfile   myfile2
$ cat myfile2
This is a test

In this case we could move files around, but we couldn’t move a file to a privileged folder, as that would need administrator rights. In a well secured environment, the damage that Shellshock can cause should be minimal, as most of the important operations require a higher-level privilege. It is this attribute of Shellshock that highlights that this is not another Heartbleed, as Heartbleed allowed anyone to access the privilege area of memory on the server, without any restrictions. While Web servers may be safe, with a limited usage of cgi-bin scripts (which allow privileged access to the system), there may be risks with poorly secured embedded systems, which can often use scripts to setup their services.

Buffer overflows and underruns

The flaw within Bash, shows how sloppy software developers have been in the past, and it is a flaw which has existed for over 25 years without being discovered. Many of the problems being under covered have been caused by poor software coding in the C++ programming language, which often allows programs to act incorrectly when the input data is not formatted as expected. Once common method of exploiting a C++ program is a buffer overflow, where a certain amount of memory is allocated to variables, and where the user enters data which is more than the allocated memory, and which causes other parts of the memory to be overwritten, and cause the program to act incorrectly.

In the case of Heartbleed it was a buffer underrun which caused the problem, where an area of memory was read and which did not actually contain the required amount of data to fill it. If you are interested in Heartbleed, and its cause (OpenSSL):

Conclusions

So after years of Microsoft Windows being the target, hackers are turning the skills onto the Linux operating system, and Mac OS is a target of choice for the end user. The flaws found in iWorm and Shell Shock show that Linux is perhaps not as rock-solid from a security point of view as many thought. With its C++ code infastructure, and lots of code which are now over a decade old, we are likely to see increasing threats on Linux, and thus to Apple devices.

To their credit, Apple have been fast to detect, and patch, but there is a danger in users not paching their systems on a regular basis. This often happened on Microsoft Windows, were users often disabled the auto-updates, typically when it caused problems on their computer. So, Apple must make sure that their users keep their systems up-to-date, otherwise Apple users may be more of a risk than Windows ones.

Apple have been stung a little with the recent security threats, but have come out fighting, and show some guts to take security seriously. One key area will be for them to find their own vulnerabilities, and not leave it solely to the community, and where there is a race to between then exploiters and the patchers. The lack of a lock-out on the “Find My iPhone” service shows that they need to take penetration testing seriously for all the products.

Apple has also been fairly immune from pirated apps, but these will increasingly become a target, as they can used as a trojan to download a whole lot of threats to the computer, including reconfiguring the system to pose threats to users. It must be remembered that much of the core of a Mac is a powerful Linux server that can be configured in the same way as any other server.

 

Two Sides of the Argument: For Every Snowden there’s a Dread Pirate Rogers

Introduction

The battle between the civil rights to privacy against the rights of society to protect itself shows no signs of abating, with the FBI saying that they are concerned about Google and Apple’s move to encrypted data by default. Unfortunately, we created file systems and file content types which had little thought on keeping things private, and where systems are often viewed as stand-along machines. We also created an Internet which is full of the same protocols that we used in the days of text terminals and mainframe computers, where users typed in commands to access data, and where there was little thought about protecting the data as it is stored, analysed and transmitted. As we are increasingly move mobile, we are now carrying around our sensitive data, that at one time was protected behind physical firewalls, and the risks to our data increases by the day. To overcome this, Apple have just released their file encryption system for iOS 8, and Google plan to do the same for the next version of Android.

The FBI, though, see the status quo as a way of investigating criminals and terrorists, but can see this opportunity reducing with encryption-by-default, such as with the file encryption system used in Apple’s iOS 8. With iOS 8, there is no encryption keys, and thus the encryption method breaches current laws, which force users to reveal their encryption keys when requested by law enforcement investigators. This would mean that users may be breaching current laws in both the US and the UK. The same battle too exists with Tor, where law enforcement are scared that crime can go un-noticed, whereas privacy advocates promote the rights of privacy of using Tor. There is thus a battle ranging from the file system to the data transmitted over the Internet.

The following outlines some key features in disk encryption:

Exception to the 5th Ammendment Right

In the UK, citizens have the right to silence (a Fifth Amendment Right in the US – related to the right against self-incrimination) but there is an exception to this related to encryption keys, and the failure to reveal encryption keys can often be seen as a sign that someone has something to hide, and is covered by Section 49 of RIPA. The move by Apple and Google may thus breach law as they must be able to hand-over their encryption key when required. This was highlighted in 2014 when Christopher Wilson, from Tyne and Wear was jailed when he refused to hand encrypted passwords related to investigations related to an attack on the Northumbria Police and the Serious Organised Crime Agency’s websites. He handed over 50 encrypted passwords, but none of these worked, so a judge ordered him to provide the correct one, but after failing to do this, he received a jail sentence of six months.

In 2012, Syed Hussain and three other men, were jailed for discussing an attack on a TA headquarters using a home-made bomb mounted on a remotely controlled toy car. Syed, who admitted have terrorist sympathises, was jailed for an additional four months for failing to hand-over a password for a USB stick.

The opposing sides

As we move into an Information Age, there is a continual battle on the Internet between those who would like to track user activities, to those who believe in anonymity. The recent Right to be forgotten debate has shown that very little can be hidden on the Internet, and deleting these traces can be difficult. The Internet, too, can be a place where crime can thrive through anonymity, so there is a continual tension between the two sides of the argument, and, overall, no-one has a definitive answer to say which is correct.

To investigation agencies the access to Internet-based information can provide a rich source of data for the detection and investigation of crime, but they have struggled against the Tor (The Onion Network) network for over a decade. Its usage has been highlighted over the years, such as when, in June 2013, Edward Snowden, used it to send information on PRISM to the Washington Post and The Guardian. This has prompted many government agencies around the World to prompt their best researchers to target cracking it, such as recently with the Russian government offering $111,000.

At the core of Tor is its Onion Routing, which uses subscriber computers to route data packets over the Internet, rather than use publically available routers. One thing that must be said is that Tor aims to tunnel data through public networks, and keep the transmission of the data packets safe, which is a similar method that Google uses when you search for information (as it uses the HTTPS protocol for the search).

The battle of the Gods

With the right to be anonymous at its core, the Tor project created a network architecture which anonymized both the source of network and the identity of users. With some many defence agencies around the World targeting Tor, the cracks have been starting to be exposed, in the same way that there has been on the targeting of OpenSSL and TrueCrypt. For this researchers identified an underlying flaw in Tor’s network design, and which has led the Tor Project has warned that an attack on the anonymity network could have revealed user identities.

This message was in response to the work of two researchers from Carnegie Mellon University (Alexander Volynkin and Michael McCord) who exploited the infrastructure. At present SEI has a Defense Department until June 2015, and is worth over $110 million a year, with a special target on finding security vulnerabilities.

Overall the attacks ran from January 2014, and were finally detected and stopped on 4 July 2014. In response to the vulnerability being found the Tor team, in a similar way to the OpenSSL Heartbleed announcement, where informed that the researchers were to give a talk at the Black Hat hacker conference in Las Vegas. The sensitives around the area is highlight by the fact that the talk was cancelled, due to neither the university nor SEI (Software Engineering Institute) approving the talk. The Tor project, through Roger Dingledine blog entry on 4 July 2014, revealed that identities could have been revealed over the period of the research.

Tor

The Web traces a wide range of information, including user details from cookies, IP addresses, and even user behaviour (with user fingerprints). This information be used to target marketing to users, and also is a rich seem of information for the detection and investigation of crime. The Tor network has long been a target of defence and law enforcement agencies, as it protects user identity and their source location, and is typically known as the dark web, as it is not accessible to key search engines such as Google. Obviously Tor could be used to bind to a server, so that the server will only talk to a client which has been routed through the Tor network, which would mean than search engines will not be able to find the content on them. This is the closed model in creating a Web which cannot be accessed by users on the Internet, and only by those using Tor. If then users trade within the dark web servers with Bitcoins, there will be little traces of their transactions.

With the Tor network, the routing is done using computers of volunteers around the world to route the traffic around the Internet, and with ever hop the chances to tracing the original source becomes reduces. In fact, it is rather like a pass-the-parcel game, where game players randomly pass to others, but where eventually the destination receiver will eventually receive the parcel. As no-one has marked the parcel on its route, it’s almost impossible to find out the route that the parcel took.

The trace of users access Web servers is thus confused with non-traceable accesses. This has caused a range of defence agencies, including the NCA and GCHQ, to invest methods of compromising the infrastructure, especially to uncover the dark web. A strange feature in the history of Tor is that it was originally sponsored by the U.S. Naval Research Laboratory (which had been involved in onion routing), and its first version appeared in 2002, and was presented to the work by Roger Dingledine, Nick Mathewson, and Paul Syverson, who have since been named, in 2012, as one of Top 100 Global Thinkers. It since received funding from Electronic Frontier Foundation, and is now developed by The Tor Project, which is a non-profit making organisation.

Thus, as with the Rights to remain private, there are some fundamental questions that remain, and it a target for many government around the World. In 2011, it was awarded the Free Software Foundation’s 2010 Award for Projects of Social Benefit for:

"Using free software, Tor has enabled roughly 36 million people around the world to experience 
freedom of access and expression on the Internet while keeping them in control of their privacy 
and anonymity. Its network has proved pivotal in dissident movements in both Iran and more recently 
Egypt."

Figure 1 shows a Web browser application setup for Tor. It uses onion routing and also the HTTPS protocol to secure the accesses. With Tor, too, the path between the two communicating hosts is also encrypted, which creates a tunnel between them. To focuses more on the security of the communication over the Internet, and less on the preserving the anonymity of the user. It is, though, often used for proxy accesses to systems, where a user wants to hide their access.

Figure 1: Tor Web browser

Silk Road

One of the first of large-scale illegal uses on Dark Web was Silk Road (created Feb 2011) by “Dread Pirate Rogers” and which was used to trade drugs on-line. In June 2011 it was pin-pointed by chatter on the Internet and for increases in Web traffic, and was taken down by the DEA and Department of Justice in the US. It has since resurfaced as Silk Road 2.0, with other similar sites appearing, along with encrypted versions of the code from the site being created so that the site can be distributed to other places, if it is taken down. This approach is equivalent to self-healing Web sites, where the re-build themselves when they are attacked. In this case, a human helper will normally be involved in re-creating the site.

While Tor had been created for all the best of reasons, from another point-of-view, it can be seen as a place that criminals can build their businesses in the Cloud, and provide a place where there can be few traces left of their activities. Overall it’s an impossible debate to say exact which is the right approach. From a law enforcement point-of-view, there are problems in investigating sites bound into the Tor network, but it also it is also a place where citizens have the rights to privacy.

Conclusions

With data breaches rising by the day, such as with 150 million passwords cracked with the Adobe infrastructure and over 120 million credit card details skimmed for Home Depot and Target, Apple and Google feel they have to build up trust with their users in their operating system. For this they are looking at encryption-by-default, where they encrypt file data (which is now stored on flash memory), and which now may breach the laws around reveal encryption keys. At one time, investigators could extract the memory from the device, and decode its contents, but without encryption keys this will be difficult. While Google and Apple have not responded to the dilemma, there could be the opportunities for them to work with the companies to overcome of the issues, which might reduce privacy settings on their data. Unfortunately if they do reduce the security on the encrypted data, they may leave open opportunities for others to learn the methods, and compromise the whole system. In a corporate market, Microsoft BitLocker is one of the most popular methods used for complete disk encryption. With this, there is always the back-door input into the encrypted data, by storing the encryption keys within the domain controller for the company.

As yet, Google or Apple have not made any comments about the issues that they encrypted file system could cause mobile phone users.

Shellshock – It is serious, but it is no Heartbleed

Introduction

After years of Microsoft Windows operating system vulnerabilities, we find that the new place which has moved to discovering sloppy programming in Java, Abode Flash, and Adobe Reader, and now Linux – with a common denominator is often around the C++ programming language. The issues around the Linux environment, causes Apple users problems, as Mac OS X has built its environment based on an underlying Linux environment.

In the days when Apple struggled to compete against Microsoft and Intel, they made two major choices:

• Move their architecture from Motorola/IBM (which was build on 68xxx processors) to the x86 architecture used by Intel. This was a smart move as it allowed them to keep up-to-date on hardware against the other PC manufacturers.
• Move their operating system to Linux. This allow them to reduce their development of their core operating system and focus on the user experience, but still use the reliability of the Linux operating system.

Unfortunately now the move to Linux might be coming back with anger on Apple, as serious flaws are now being found in the core infrastructure. Many bugs, which have gone unnoticed for many users, are being found, and these are often due to a lack of code review and testing. With OpenSSL, for example, and which caused Heartbleed, we still see C++ code developed Eric Young (“eay” lib) in 1998.

iWorm

Last week, Dr. Web (a Russian security) outlined a new worm for OS X: “Mac.BackDoor.iWorm“, and which allows hackers to take control of the computer, and using it as part of a Botnet, such as for sending spam emails or performing Denial of Service attacks against web sites. Apple, on detecting over 17,000 users with the worm then was quick to respond, and added the signature of iWorm to its malware detector (Xprotect – and which is installed on every Mac computer). A key factor for users is that there Mac needs to be up-to-date with system updates, otherwise they can be open to the spread of the worm. The Xprotect signature defined is:

This worm has been used to search Reddit for a fake discussion forum related to Minecraft and also integrated links to hacker controlled command servers. Reddit then closed the hacker accounts that were used in order to share links to the commands servers, and banned the fake Minecraft subreddit – which stopped the worm from receiving orders from the hackers. There are also rumours that the worm itself was spread though pirated software, such as for Microsoft Office, Adobe Photosho pand Parallels (the virtualisation package used to run other operating systems on the Mac) from The Pirate Bay.

Shell Shock

With Shell Shock the target was again Linux hosts using Bash (GNU Bourne Again Shell), which is the command line interpreter used in many Linux based systems, including Apple OS X. Bash interprets the commands that users enter or are run from scripts, and then makes calls to the operating system, such as for running programs, listing the contents of a directory, or in deleting files. The discovered flaw allows intruders to remotely run arbitrary code on systems such as Linux servers including for web servers, routers, and many embedded systems. It was discovered by Stephane Chazelas of Akamai, who found that code at the end of a function of an exported variable is run whenever an environment variable is used within the Bash environment. Many Linux programs use environmental variables to pass parameters between programs, and the flaw thus allows for code to be inserted into a program whenever these environmental variables are called.

Shellshock focuses on CGI scripts, which are old-fashioned scripts that allow commands to be processed using a scripted language. While popular in the past, it has been largely replaced by PHP and other high-level scripting programs. In most cases CGI scripts reside in the /cgi-bin folder. For GNU Bash through 4.3, trailing strings after a function are processed in the definitions of environment variables. This allows intruders to execute arbitrary cod. For example, we have a function named mybugtest:

billbuchanan@Bills-MacBook-Pro:/tmp$ export mybugtest='() { :;}; echo I AM BUGGY'
billbuchanan@Bills-MacBook-Pro:/tmp$ bash -c "echo Hello"
 I AM BUGGY
 Hello

Shell Shock can comprise a system by injecting a payload of code into the environment variables of a running process. When the process is started, the code is injected into the running program, in the same way as a user typing in some user input.

The code which can appear at the end of the Bash function can be fairly complex, and allow an intruder to inject code into the shell (and thus into running programs). In this example we copy some text into a text file (named newfile) and then copy the file to a new file (newfile2):

$ export mybugtest='() { :;}; echo "This is my new file..." > newfile; cp newfile newfile2'
$ bash -c ""
Segmentation fault: 11
$ ls myf*
myfile   myfile2
$ cat myfile2
This is a test

In this case we could move files around, but we couldn’t move a file to a privileged folder, as that would need administrator rights. In a well secured environment, the damage that Shellshock can cause should be minimal, as most of the important operations require a higher-level privilege. It is this attribute of Shellshock that highlights that this is not another Heartbleed, as Heartbleed allowed anyone to access the privilege area of memory on the server, without any restrictions. While Web servers may be safe, with a limited usage of cgi-bin scripts (which allow privileged access to the system), there may be risks with poorly secured embedded systems, which can often use scripts to setup their services.

Buffer overflows and underruns

The flaw within Bash, shows how sloppy software developers have been in the past, and it is a flaw which has existed for over 25 years without being discovered. Many of the problems being under covered have been caused by poor software coding in the C++ programming language, which often allows programs to act incorrectly when the input data is not formatted as expected. Once common method of exploiting a C++ program is a buffer overflow, where a certain amount of memory is allocated to variables, and where the user enters data which is more than the allocated memory, and which causes other parts of the memory to be overwritten, and cause the program to act incorrectly.

In the case of Heartbleed it was a buffer underrun which caused the problem, where an area of memory was read and which did not actually contain the required amount of data to fill it. If you are interested in Heartbleed, and its cause (OpenSSL):

Conclusions

So after years of Microsoft Windows being the target, hackers are turning the skills onto the Linux operating system, and Mac OS is a target of choice for the end user. The flaws found in iWorm and Shell Shock show that Linux is perhaps not as rock-solid from a security point of view as many thought. With it’s C++ code infrastructure, and lots of code which are now over a decade old, we are likely to see increasing threats on Linux, and thus to Apple devices. To their credit, Apple have been fast to detect, and patch, but there is a danger in users not patching their systems on a regular basis. This often happened on Microsoft Windows, we users often disabled the auto-updates, typically when it caused problems on their computer. So, Apple must make sure that users keep their systems up-to-date, otherwise Apple users may be more of a risk than Windows ones.