Mr Bates vs The Post Office
Discussion
dmsims said:
One of the (many) damning issues is that no one ever thought to trace where the "money" had gone. (I know this wasn't a requirement)
Quite, no attempt to look at bank or credit card records, unexplained expenditure on houses, holidays, cars, food etc. Just "you're guilty, look at what Horizon says".LimmerickLad said:
dmsims said:
One of the (many) damning issues is that no one ever thought to trace where the "money" had gone. (I know this wasn't a requirement)
Easier to blame the "thieving" SPM's I suppose.It's still utterly shocking, isn't it.
LimmerickLad said:
Bonefish Blues said:
siremoon said:
LimmerickLad said:
I'm probably getting way out of my depth here so apologies if this doesn't make sense:
IIRC a lot of the dancing on the head of a pin was more about was Horizon "systemically flawed" and not about whether it was "fundamentally flawed"?
Whilst all the damage done to many SPM's is 100% unnacceptable and tragic and heads should roll for it, are they not saying that, given that 99.999% of transactions are fine and accepting the system has some "fundamental flaws" (i.e. faults & weaknesses), but the system is not "systemically flawed" because it works fine 99.999% of the time on millions of transaction, however it was what happened (or didn't) after these "flaws" were found that is the real problem.
Imo the system was systemically flawed because it allowed unaudited (or inadequately audited) changes to the accounts via a back door. That back door wasn't a bug, it was a deliberately implemented feature. The bugs related to the incorrect calculation of the numbers brought that systemic flaw into play because the back door was used in an attempt to hide the effect of the bugs without the knowledge of one set of key stake holders ie the SPMs. The number of accounting bugs and the use of the back door to manipulate the consequences of those bugs created the conditions for what happened subsequently. IIRC a lot of the dancing on the head of a pin was more about was Horizon "systemically flawed" and not about whether it was "fundamentally flawed"?
Whilst all the damage done to many SPM's is 100% unnacceptable and tragic and heads should roll for it, are they not saying that, given that 99.999% of transactions are fine and accepting the system has some "fundamental flaws" (i.e. faults & weaknesses), but the system is not "systemically flawed" because it works fine 99.999% of the time on millions of transaction, however it was what happened (or didn't) after these "flaws" were found that is the real problem.
The very fact that the PO and Fujitsu consistently lied about the existence of the back door tells you they knew it wasn't something that should exist in the form it did in a system of this nature.[/b]
Edited by siremoon on Sunday 30th June 08:07
Having a backdoor purposely built in kind of makes sense........it's the fact it was hidden, denied existance and access not recorded was the issue, therefore as the system worked 99.999% of the time, the system in itself was fine but it was the arsholes involved in the "cover ups" to hide its use and existance that are to blame not the system itself? I tend to see the backdoor as a bit of a built in engine management code reader that allows you to see what went wrong and then reset once the fault was fixed / rectified but happy to be corrected if I am seeing it wrongly.
I live by the motto - it isn't what you do wrong but what you do to put it right that matters.......the problem as I see it was once the 1st PM was prosecuted for something that was actually down to the faulty system, in POL & FJ's minds they couldn't admit the "back door" hence the whole lie just snowballed and, aagin IMO, became a conspiracy and a coverup at the expense of the small people in this i.e. the SPM's..those involved in this conspracy should pay a very high price IMO but I have a funny feeling there will only be 1 or 2 scapegoats and the real villains PV, JS et al and the lawyerswill get away with it scott free!
What should have happened is that in the prosecutions, there should have been evidence submitted which showed that i) remote access was possible, and ii) there had been no use of remote access during the relevant time. However, not even FJ had this evidence (see the Horizon Issues trial). This is what made the prosecutions fundamentally unsound. If any defence had asked whether remote access was possible, then FJ/PO should have owned up and said 'yes', but they didn't. I'm not aware of anyone saying that a shortfall was caused directly as a result of remote access having been used, and I'm not sure the CCRC did either (see the Hamilton CoA judgment).
I could imagine that GJ's view was that remote access was there as a tool; if it was ever used, it was only used to fix things so would never give rise to something that would penalise a SPM, so why would it be relevant in a case where there was a problem?
onetwothreefour said:
I think I agree with the second portion above in bold - the existence of the possibility of super-user remote access per se wasn't an issue (and I'm not sure it is today), it is the fact that it was not logged, not auditable and there were no/weak controls about how it was used, combined with the potential to modify the underlying data, meant that any evidence that was derived from the underlying data (including ARQs etc.) should have come with the caveat that that data could not be guaranteed to represent the actions of the SPM. (I'm not sure that the existence of a backdoor weakens a system provided that you can show what, if anything, it was ever used for).
What should have happened is that in the prosecutions, there should have been evidence submitted which showed that i) remote access was possible, and ii) there had been no use of remote access during the relevant time. However, not even FJ had this evidence (see the Horizon Issues trial). This is what made the prosecutions fundamentally unsound. If any defence had asked whether remote access was possible, then FJ/PO should have owned up and said 'yes', but they didn't. I'm not aware of anyone saying that a shortfall was caused directly as a result of remote access having been used, and I'm not sure the CCRC did either (see the Hamilton CoA judgment).
I could imagine that GJ's view was that remote access was there as a tool; if it was ever used, it was only used to fix things so would never give rise to something that would penalise a SPM, so why would it be relevant in a case where there was a problem?
I think that's a good summary. The support people have said that the remote access was logged when they (rarely) used it as part of fixing a case, but there seems to have been a way of doing it without getting logged, which is obviously a serious problem for court evidence, even if it never happened in practice. I think the dramatisations and media coverage have led people to believe that it's proven that Fujitsu were in there "tampering" all the time, but there's really been no evidence of that.What should have happened is that in the prosecutions, there should have been evidence submitted which showed that i) remote access was possible, and ii) there had been no use of remote access during the relevant time. However, not even FJ had this evidence (see the Horizon Issues trial). This is what made the prosecutions fundamentally unsound. If any defence had asked whether remote access was possible, then FJ/PO should have owned up and said 'yes', but they didn't. I'm not aware of anyone saying that a shortfall was caused directly as a result of remote access having been used, and I'm not sure the CCRC did either (see the Hamilton CoA judgment).
I could imagine that GJ's view was that remote access was there as a tool; if it was ever used, it was only used to fix things so would never give rise to something that would penalise a SPM, so why would it be relevant in a case where there was a problem?
The issue with the remote access via FJ helpdesk was it was at first denied and then when confirmed it was shown to be access that was made under the SPMs account so any and all keystrokes would have been recorded as coming from the SPM.
That is completely unacceptable in a banking and money handling environment, like having Barclays help desk staff with the ability to access your online bank account under your credentials utterly barmy.
No conviction was safe after that was revealed, and that's before you look at the bugs that were causing the fictional losses.
Wills2 said:
The issue with the remote access via FJ helpdesk was it was at first denied and then when confirmed it was shown to be access that was made under the SPMs account so any and all keystrokes would have been recorded as coming from the SPM.
That is completely unacceptable in a banking and money handling environment, like having Barclays help desk staff with the ability to access your online bank account under your credentials utterly barmy.
No conviction was safe after that was revealed, and that's before you look at the bugs that were causing the fictional losses.
That’s the gist of it. From a Fujitsu POV, I can get that they might genuinely say “yes but we don’t use it like that.”That is completely unacceptable in a banking and money handling environment, like having Barclays help desk staff with the ability to access your online bank account under your credentials utterly barmy.
No conviction was safe after that was revealed, and that's before you look at the bugs that were causing the fictional losses.
The bottom line is Horizon seems to have been built by people with no real understanding of such systems. Even the descriptions of “double entry” are wrong.
They’d have been far better off licensing or acquiring, say, TopManage and building on top of that - not only for the solid fundamentals, but because they’d have had a solid commercial ERP offering to sell elsewhere.
If the fundamental data and system architecture had been right, none of the rest would or could have happened.
It's a scandal within a scandal, anyone with even the briefest of experience of ERP systems (not even in development) but just as a manager within the environments where they are deployed would know that you cannot set a system up like that, the fact that FJ did is astounding.
But none of that is an excuse for what happened as a consequence of the lack of audit and control, what followed was a criminal conspiracy across a broad spectrum players and that is the real scandal, people wilfully doubled down on that lack of audit and control and used it for their own ends whilst destroying lives.
skwdenyer said:
That’s the gist of it. From a Fujitsu POV, I can get that they might genuinely say “yes but we don’t use it like that.”
The bottom line is Horizon seems to have been built by people with no real understanding of such systems. Even the descriptions of “double entry” are wrong.
They’d have been far better off licensing or acquiring, say, TopManage and building on top of that - not only for the solid fundamentals, but because they’d have had a solid commercial ERP offering to sell elsewhere.
If the fundamental data and system architecture had been right, none of the rest would or could have happened.
Well, they did really do that with the original Horizon, which was built on top of Escher Riposte (https://www.eschergroup.com/riposte-platform/). That iteration of Horzion had the most issue, although not all of them were to do with Riposte. The bottom line is Horizon seems to have been built by people with no real understanding of such systems. Even the descriptions of “double entry” are wrong.
They’d have been far better off licensing or acquiring, say, TopManage and building on top of that - not only for the solid fundamentals, but because they’d have had a solid commercial ERP offering to sell elsewhere.
If the fundamental data and system architecture had been right, none of the rest would or could have happened.
Then there was the second iteration of Horizon (Horizon Online) which removed Riposte.
hidetheelephants said:
dmsims said:
One of the (many) damning issues is that no one ever thought to trace where the "money" had gone. (I know this wasn't a requirement)
Quite, no attempt to look at bank or credit card records, unexplained expenditure on houses, holidays, cars, food etc. Just "you're guilty, look at what Horizon says".1. Why is there suddenly a huge spike in fraud? and
2. Where's the money gone?
Panamax said:
hidetheelephants said:
dmsims said:
One of the (many) damning issues is that no one ever thought to trace where the "money" had gone. (I know this wasn't a requirement)
Quite, no attempt to look at bank or credit card records, unexplained expenditure on houses, holidays, cars, food etc. Just "you're guilty, look at what Horizon says".1. Why is there suddenly a huge spike in fraud? and
2. Where's the money gone?
2. Not so much - such a lack of curiosity.
Panamax said:
This is IMO why the system itself is just a side-show in the context of gross management failings at PO. Two fundamental questions should have been front and centre at PO,
1. Why is there suddenly a huge spike in fraud? and
2. Where's the money gone?
Bear in mind that, as I understand it, for false accounting, you didn't have to have actually taken the money, just misrepresented the accounts (although even that was more complicated because in reality the SPMs had no option to dispute the accounts they were submitting).1. Why is there suddenly a huge spike in fraud? and
2. Where's the money gone?
For Q1, as said above, pre-Horizon, the PO suspected that large amounts of fraud were going on that they couldn't detect. Now, they could and there it was!
I think the entire thing can be summed up by looking at Angela VDB's evidence at the Horizon issues trial: SPM "I did this and this and this and this and then this happened". AVDB "But it shouldn't have". POL didn't dare look into what had actually happened, because they knew (or feared, either way it doesn't matter) that the SPM would be found to be correct. At the end, POL essentially had to disagree with its own expert (Godeseth) who basically agreed that what the SPM had said was plausible.
Whether AVDB knew the SPM was correct or whether she didn't know but didn't want to find out, the fact that either one of those was the case was dreadful. This was the end of an avalanche that started with a shoddy prosecution system in a culture where POL was correct and the SPMs were wrong. "Incurious" came up a lot, and while you could possibly excuse some people (GJ whose day job was development/code/etc. being incurious about incomplete legal formalities in a document given to him by a lawyer), there are others where it clear they were just not doing what they should have been doing, and I suspect, will be found that their omission was contrary to legal or regulatory requirements that set out how they should be doing their jobs.
CharlesElliott said:
skwdenyer said:
That’s the gist of it. From a Fujitsu POV, I can get that they might genuinely say “yes but we don’t use it like that.”
The bottom line is Horizon seems to have been built by people with no real understanding of such systems. Even the descriptions of “double entry” are wrong.
They’d have been far better off licensing or acquiring, say, TopManage and building on top of that - not only for the solid fundamentals, but because they’d have had a solid commercial ERP offering to sell elsewhere.
If the fundamental data and system architecture had been right, none of the rest would or could have happened.
Well, they did really do that with the original Horizon, which was built on top of Escher Riposte (https://www.eschergroup.com/riposte-platform/). That iteration of Horzion had the most issue, although not all of them were to do with Riposte. The bottom line is Horizon seems to have been built by people with no real understanding of such systems. Even the descriptions of “double entry” are wrong.
They’d have been far better off licensing or acquiring, say, TopManage and building on top of that - not only for the solid fundamentals, but because they’d have had a solid commercial ERP offering to sell elsewhere.
If the fundamental data and system architecture had been right, none of the rest would or could have happened.
Then there was the second iteration of Horizon (Horizon Online) which removed Riposte.
From what I’ve gleaned, what happened was Post Office were sold Riposte, using the AnPost implementation as a reference. They then decided to extend that to meet the original joint PO/DSS contract requirement. Along the way, Riposte moved from an old Windows 3.11 system to something based off of NT. The PO scope was fundamentally different - and unsuited - to Riposte, but PO at this stage were wedded to this system “designed for postal services” and so mandated the continual shoehorning of their requirements into the space provided by Riposte. One of the early meeting documents makes it clear that (a) Riposte is now a complete different beast to the original, and (b) the various participants don’t work together and don’t understand the complete architecture.
By the end, they seem to have a product that started as a design for a milking stool and ended with orbital capability - without ever re-evaluating whether the milking stool remained a suitable base.
ICL/Fujitsu should probably have said “if you want us to stand by this product, we need to make all the architectural choices.” It doesn’t seem that that happened.
So we end up with ICL/Fujitsu as principal contractor, reliant upon underpinnings provided by a third party that were mandated by the customer, but without access to the source code of Riposte.
That’s not what I suggested above. They didn’t licence Riposte; it seems they simply bought instances of Riposte and attempted to integrate it with other components.
Rather than call Horizon software, it would be more honest to call it a contract. Jenkins, for instance, as “expert” had no ability to lift the lid on Riposte’s source code or assert its fitness - he’d just inherited a bunch of mashed-together first and third party code with (we’ve learned elsewhere) little or no proper testing.
As regards remote access, etc, either Riposte was insecure (possibly - for instance, in the An Post implementation it relied on plain text transaction log files FFS), or the PO implementation and use simply allowed ICL/Fujitsu folk to RDP in (other technologies are available) and perform tasks (this is born out by other evidence, which said if a counter user wasn’t logged in then logging was off - this fits with the Riposte architecture requiring mag stripe card user log in to initialise transaction sequencing and so on - I do wonder if sometimes they really did just hack the local files).
Horizon wasn’t - software system; it was an IT system. Because Riposte wasn’t necessarily fit for the purpose, there were too many bolt-ons and workarounds.
I maintain that, if they’d started with a proper extensible ERP or similar system, and then worked properly to build an architecture that didn’t break the basics of system audit, they’d have had a far more robust end result.
But who is to blame? Probably a mixture - PO for doing as so often happens and demanding the software they’ve been sold, and ICL/Fujitsu for saying yes.
I wonder what the original contract says?
This is all degrees for me. Was the software bullet proof? Certianly not, but in the 2000s, building a distributed system across 100s of branches with dodgy network connections and non-computer literate users was even harder that it would be today. Overall, it worked pretty well. People - most of whom didn't understand IT either - incorrectly used the fact that it worked 'most of the time' to believe that it worked 'all of the time'.
Then we get to Fujitsu who were protecting their contract and not being upfront when the pressure started. The Post Office investigators knew nothing about IT or investigating, but they found a way to ensure convictions or recovery and that was their personal measure of success. Gareth Jenkins was a godsend as he was an expert in certain parts of the system and could explain them in detail, but crucially, he was prepared to extend his knowledge of specific areas to make general statements of reliability that would have appeared extremely credible. The senior people didn't want to hear that they might be sitting on years of bad management decisions so started to look the other way, become obstructive, and avoid anything that could have got to the truth. That unravelled when they picked Second Sight and it didn't got the way they expected.
Horizon - the software - wasn't great, but I think it was the least bad element of the whole story. I mean, some software was rolled out to 100s of post masters and sometimes it didn't work - that really isn't a surprise to anyone, is it? It's what happened from that point which was the real issue.
Then we get to Fujitsu who were protecting their contract and not being upfront when the pressure started. The Post Office investigators knew nothing about IT or investigating, but they found a way to ensure convictions or recovery and that was their personal measure of success. Gareth Jenkins was a godsend as he was an expert in certain parts of the system and could explain them in detail, but crucially, he was prepared to extend his knowledge of specific areas to make general statements of reliability that would have appeared extremely credible. The senior people didn't want to hear that they might be sitting on years of bad management decisions so started to look the other way, become obstructive, and avoid anything that could have got to the truth. That unravelled when they picked Second Sight and it didn't got the way they expected.
Horizon - the software - wasn't great, but I think it was the least bad element of the whole story. I mean, some software was rolled out to 100s of post masters and sometimes it didn't work - that really isn't a surprise to anyone, is it? It's what happened from that point which was the real issue.
CharlesElliott said:
This is all degrees for me. Was the software bullet proof? Certianly not, but in the 2000s, building a distributed system across 100s of branches with dodgy network connections and non-computer literate users was even harder that it would be today. Overall, it worked pretty well. People - most of whom didn't understand IT either - incorrectly used the fact that it worked 'most of the time' to believe that it worked 'all of the time'.
Then we get to Fujitsu who were protecting their contract and not being upfront when the pressure started. The Post Office investigators knew nothing about IT or investigating, but they found a way to ensure convictions or recovery and that was their personal measure of success. Gareth Jenkins was a godsend as he was an expert in certain parts of the system and could explain them in detail, but crucially, he was prepared to extend his knowledge of specific areas to make general statements of reliability that would have appeared extremely credible. The senior people didn't want to hear that they might be sitting on years of bad management decisions so started to look the other way, become obstructive, and avoid anything that could have got to the truth. That unravelled when they picked Second Sight and it didn't got the way they expected.
Horizon - the software - wasn't great, but I think it was the least bad element of the whole story. I mean, some software was rolled out to 100s of post masters and sometimes it didn't work - that really isn't a surprise to anyone, is it? It's what happened from that point which was the real issue.
All very good points. For those of us who've been knocking around "serious IT" for a long time, I'd say one of the big problems is they were trying to do all of this with first Windows 3.11 and then later NT 3.5.1 / 4, and were clearly inventing the wheel over a bunch of asynchronous data transfer stuff - not uncommon for the PC boys back in the day.Then we get to Fujitsu who were protecting their contract and not being upfront when the pressure started. The Post Office investigators knew nothing about IT or investigating, but they found a way to ensure convictions or recovery and that was their personal measure of success. Gareth Jenkins was a godsend as he was an expert in certain parts of the system and could explain them in detail, but crucially, he was prepared to extend his knowledge of specific areas to make general statements of reliability that would have appeared extremely credible. The senior people didn't want to hear that they might be sitting on years of bad management decisions so started to look the other way, become obstructive, and avoid anything that could have got to the truth. That unravelled when they picked Second Sight and it didn't got the way they expected.
Horizon - the software - wasn't great, but I think it was the least bad element of the whole story. I mean, some software was rolled out to 100s of post masters and sometimes it didn't work - that really isn't a surprise to anyone, is it? It's what happened from that point which was the real issue.
There's nothing they were trying to do on the distributed side that hadn't be solved thousands of times before with more robust operating systems and tools
They (Escher) seem even to have developed their own messaging system, rather than just using something like TIB or even MQ and run the whole thing on something like SCO. I'd imagine anyone actually used to this sort of work would be doing the front end in something like PowerBuilder, rather than VB.Anyhow, that's a sideshow I guess.
dmsims said:
Talking of sideshows
Why wouldn't they gone for a centralised system?
Not sure entirely what you mean by centralised, but this was in the early 2000s when connectivity was patchy. If it was working, terminals in branches would upload everything to a central server, but they were programmed to be able to work if connectivity was down. They would store transactions locally and then upload them centrally when connectivity was restored. They could also cope with one or more terminals in a branch going down, and sync'ing transactions between terminals locally.Why wouldn't they gone for a centralised system?
Centralised - one big computer! and dumb terminals
How did banks manage ?
How did banks manage ?
CharlesElliott said:
Not sure entirely what you mean by centralised, but this was in the early 2000s when connectivity was patchy. If it was working, terminals in branches would upload everything to a central server, but they were programmed to be able to work if connectivity was down. They would store transactions locally and then upload them centrally when connectivity was restored. They could also cope with one or more terminals in a branch going down, and sync'ing transactions between terminals locally.
Gassing Station | TV, Film, Video Streaming & Radio | Top of Page | What's New | My Stuff