Utopia Talk / Politics / Jergul: Nord stream

Seb Member	Wed Nov 16 02:55:02 http://www...-pipeline-explosion-dark-ships This is the kind of thing you can do. But you can also do it with coastal radar and other stuff (no idea how sparse in time the satellite footage coverage is, but it can be surprisingly high). We pitched something similar for brexit: monitoring the lorry parks around pays-de-calais in the event of no-deal to give early warnings of tail backs.
jergul large member	Wed Nov 16 05:41:15 2 of 23 vessels with transponders off sounds about right. What does not sound right is that only 23 vessels passed through the area in 90 days. I just checked. Fishing vessels are not required to use them. My argument was that satellite scrubing would take a long time, that radar data is generally not stored and that even finding a within a few nautical miles would prove nothing. The clumps of metal in motion found were too big, moving too fast, and too far away imo. The company needs to work on its algorythms. It missed things.
Seb Member	Wed Nov 16 06:21:51 jergul: "My argument was that satellite scrubing would take a long time" It really wouldn't. "that radar data is generally not stored" Not true - even from a research or commercial perspective you can buy historical HFR data files. EU operates a network for research purposes and I would be flabbergasted if the defence and security services do not maintain it for precisely this kind of purpose.
jergul large member	Wed Nov 16 06:30:52 Seb It took a long time and proved nothing. What part of the word "generally" do you not understand?
jergul large member	Wed Nov 16 06:35:03 https://repository.oceanbestpractices.org/bitstream/handle/11329/1451/SOP_HFR_guidelines-on-how-to-sync-your-high-frequency-%28hf%29-radar-data-with-the-european-hf-radar-node_v1.2%281%29.pdf?sequence=1&isAllowed=y There is the easy to use process.
jergul large member	Wed Nov 16 06:35:13 http://rep...%29.pdf?sequence=1&isAllowed=y
Seb Member	Wed Nov 16 06:46:45 jergul: I understand it to be a jergalistic caveat that renders the following statement undeterminable in any objective way, thus allowing the jergul to declare himself right under any circumstance. As I said, I'm pretty sure that most countries defence and security services will be holding tracking data. I'm not sure what you are hoping to prove by posting the integration guidance for the data pipeline into a store.
jergul large member	Wed Nov 16 07:54:40 I am quite sure many countries do keep some radar data stored for quite a few days. Lets estimate. 70% of countries keep 30% of radar data stored for at least 3 days, and 2% of that data stored for more than a month.
Seb Member	Wed Nov 16 08:10:24 Estimates have some kind of basis normally - not "numbers I have pulled up at random".
jergul large member	Wed Nov 16 08:12:58 That seems a flawed approach given their inherent inaccuracy. Casting a pseudo-scientific glaze over opinions leads to cranium measuring.
Seb Member	Wed Nov 16 09:30:07 Nope. Why 70% rather than 20% or 0% or 100% of countries? "Oh, 70% is so inaccurate so why do I need a basis". Assumptions generally come with justification.
jergul large member	Wed Nov 16 09:36:01 Generally, yes. Not in this case. Why bother with the justification? I am sure I could find something if I wanted to. BS justifications is one of the elitist tricks done to make people with long post secondary educations feel they have not wasted their time.
Seb Member	Wed Nov 16 16:30:33 If you have no justification it's just a WAG and shouldn't be put forward - as you do - as a solid basis for discussion.
Seb Member	Wed Nov 16 16:33:03 All we can say is Coastal radar data exists - historic commercial and research grade data sets exist - and there are obvious use cases for national security agencies to hold onto it for periods of time - especially in strategically important areas like the baltic.
jergul large member	Thu Nov 17 04:57:10 Seb Yah, that is one of the barriers elitist pricks like to use to bolster their own sense of usefulness. Your point is invalid unless cloaked in academic babble. How else can we distinguish the valued opinions of my ilk from worthless opinions from the rabble? You are making extraordinary claims. Lets see that coastal radar data.
jergul large member	Thu Nov 17 04:59:15 To be fair. I consider myself an elitist prick too, but I dont really care enough to promote a sense of usefulness. At least not in this case.
Seb Member	Thu Nov 17 05:56:46 Jergul: What utility does "Norwegian ex-fisherman asserts with no basis that 70% of countries keep 30% of reader radar data for 3 days" have to anyone other than you?
Seb Member	Thu Nov 17 05:58:35 Assumptions and rationale are - contrary to your assertions - anti-elitest tools to create transparency and basis for challenging bold but baseless assertions by self appointed but ultimately naked authorities.
jergul large member	Thu Nov 17 06:03:00 Seb What utility does a "British ex-preschooler..." See what I did there? :D
Seb Member	Thu Nov 17 06:14:53 Well, you wanted to make this about status, so I think you are rather proving my point. Naked assertions count for little to nothing, making them and demanding they be asserted is an implicit claim to authority. Hence, better to provide rationale and make assumptions explicit.
Seb Member	Thu Nov 17 06:15:12 *they be accepted
jergul large member	Thu Nov 17 06:25:26 Seb The act of discover and the act of justification are two established principles. Elitist pricks weigh justification heavily because they can sally forth with pseudo-scientific babble. It quite often boils down to unverified assumptions cloaked by jargon. See: "HFR data files" Frankly, FB honesty on the lines of US best, we would keep all data because we might need it to prove communists are taking over government Is more honest and actually provides a deeper justification than your assertions so far.
Seb Member	Thu Nov 17 06:27:15 Jergul: "Lets see that coastal radar data." I've never actually seen a Russian nuclear weapon pit. Have you? Do you doubt then they exist? So do we seriously doubt that various Baltic Nations militaries that have HF surveillance radar don't keep the data for a period? Poland, Sweden and Denmark all maintain military/security/police HF coastal surveillance radar systems. The idea they don't keep hold of data seems crazy. Why do you think they don't? Given the specific military purpose of coastal surveillance is to track ships that pose a threat to law and order or security, the idea they don't keep data for more than a few days is the extraordinary claim here.
jergul large member	Thu Nov 17 06:31:22 (I have completed 3 degrees since I quit fishing if that is your point. Or the same number you have since leaving pre-school:D).
Seb Member	Thu Nov 17 06:36:34 Jergul: "It quite often boils down to unverified assumptions cloaked by jargon" A verified assumption isn't an assumption. The question is whether the assumption is reasonable. Fact: Denmark, Poland and Sweden all operate military/police/security HF coastal surveillance radars with a primary purpose to track ships. Fact: This data is not terribly onerous to store: theres actually abundant historical hf radar data from civil systems available for research purposes. Assumption: The polish/Swedish/danish militaries keep radar data. This is reasonable because it is obvious that would want to be able to look at ships of interests course or identify ships of interest after an event (e.g. as part of police investigations on smuggling); and it would not be very costly to maintain such databases compared to operating the radar system in the first place. Conclusion: It would be pretty easy to identify a shortlist of ships that were involved in the bomb attacks by cross referencing a mix of AIS, satélite and radar data, amongst other things. The perpetrator has a high likelihood of being identified to high confidence by Baltic coastal States.
jergul large member	Thu Nov 17 06:38:33 Seb You say there is evidence. Prove it. You are making quite the extraordinary claim. What is the name of the information sharing programme the Estonian military uses to collect and store radar data? Or is your theory that all data is kept at radar sites forever just in case it might be needed later. In the same way CCTV data is stored forever. Not.
jergul large member	Thu Nov 17 06:40:58 All surveillance data is kept by big government forever for reasons because they want to take away our guns. There. The FB equivalent of what you just said.
Seb Member	Thu Nov 17 06:41:43 Jergul: My point is not about your qualifications or lack of them. It's the absurdity of making bald assertions and then claiming that doing so is an exercise in anti-elitism when it is nothing of the sort. "These are a valid basis for conversation you my interlocutor must accept this because I have said them, and for no other reason" is an appeal to authority. Dismissing the entire idea that such statements should come with justification as elitist is tin foil hat stuff
jergul large member	Thu Nov 17 06:47:08 Seb Your half-assed assumptions cloaked in scientific babble is an appeal to authority with an attempt at levity through leveraging a post secondary education. In sum: Your shit smells the same as mine. I frankly dont see why you bother for things you dont care much about.
Seb Member	Thu Nov 17 06:49:53 Jergul "You are making quite the extraordinary claim" What's extraordinary about it? P.s. nobody said forever. In fact I explicitly said a period. I would guess at least six months - whatever they deem a useful period of time based on their need. You suggested they would retain only 2% would be kept for a month. That's a very specific claim. Quite extraordinary. "All surveillance data is kept by big government forever for reasons because they want to take away our guns." The govt doesn't want to take away your guns, and surveillance data wouldn't help them do it. Security forces do want to track ships in coastal waters to support investigation and responses to threats - that's explicitly why they built the radars. Your analogy is more like denying police have access to a database of who has a gun in a country that operates a gun licensing system.
jergul large member	Thu Nov 17 06:51:29 Incidentally, I am arguing against myself. If data had been systematically gather and a huge smoking gun pointed towards Russia, then don't you think we would know by now? So either data has huge holes, or ships with ties to the West are suspect.
jergul large member	Thu Nov 17 06:53:05 You are saying 100% of data is kept for 6 months. A very specific and extraordinary claim. Where and how. Never mind the direct evidence. What funding does such a project have in the various countries you are claiming do it?
jergul large member	Thu Nov 17 06:54:41 Government needs to know who has what guns to be able to take them away, silly.
jergul large member	Thu Nov 17 06:57:39 Can you name one ship inquiry that accessed historical radar data to prove which vessel was in the wrong?
Seb Member	Thu Nov 17 08:27:29 Jergul: Or they don't want to escalate. "whatever they deem a useful period of time based on their need" is what I said. "A very specific and extraordinary claim." Guess. See, transparency. Why would it be extraordinary? What's extraordinary about keeping this data for six months? How many TB of data do you think the raw returns are, let alone processed ship tracks? "Government needs to know who has what guns to be able to take them away, silly." Yeah, CCTV surveillance doesn't tell you that. "Can you name one ship inquiry that accessed historical radar data to prove which vessel was in the wrong" What do you think the surveillance network is for, jergul?
jergul large member	Thu Nov 17 09:14:58 Seb "Dont want to escalate" Now that was a hell of a weak CT. Multiple government agencies are subverting justice to avoid what exactly? Whatever is deemed useful is until the local radar station purges its cache. So many times a day. How many PB? I dunno. Say 25 Mb of data per second per radar. CCTV is hardly all of the full spectrum surveillance big government is doing to take away our guns. See your list of all the surveillance assets government is surely storing for just open waters. What radars are commonly for is to aid navigation and avoid accidents. So, zero ship inquiries have determined fault by using the imaginary comprehensive historical radar data you prosume exist? Surely you can at least concoct at least one imaginary ship inquiry to go with your imaginary data storage so your fairy tale makes more sense? You are making extraordinary claims. Prove them.
Seb Member	Thu Nov 17 09:54:38 jergul: " Multiple government agencies are subverting justice to avoid what exactly?" It's not a question of internal justice systems though, this would be a security matter and in any case the jurisdiction here is murky if it is Russia blowing up a pipeline owned by er.. Russia in international waters. In any case isn't your argument that we all know it is Ukraine and are covering it up? "Whatever is deemed useful is until the local radar station purges its cache." Oh. You think the data isn't integrated and stays only with local stations. "Say 25 Mb of data per second per radar" - based on? In any case, you wouldn't need or want to take the raw level 0 data - you'd take the analysed and filtered data. This is an absurd number. "What radars are commonly for is to aid navigation and avoid accidents." Mate, the ITTs for the systems I'm talking about are OJEU'd and their manufacturers and integrators issue press releases about how great they are at tracking pirates and smugglers and other such. They are for detecting and tracking ships, boats and low flying light aircraft. "zero ship inquiries" I have no idea and no inclination to look for it as it is pointless. But it would be extraordinary if the systems that these three countries have procured and operate specifically to track ships and boats for law enforcement and security purposes do not, as you claim, actually do that but just provide real-time only information. "imaginary data storage" Well, you can download historical data from numerous civil systems. The idea that each local radar station just throws all the data away instantly is absurd. Simply keeping a log of all contacts and their movements isn't going to be remotely 25Mb/s per station.
jergul large member	Thu Nov 17 10:17:09 Seb Are you suggesting a crime was not committed, that it is being investigated as a crime, but that subverting justice by mulitiple agencies in numerous countries is still taking place. Because nobody want politicians to have to chose to not escalate? Why not then at least have a coherent CT and conclude that your fairytale evidence suggests Western/Ukrainian assets are behind the detonation and it is being kept secret to not undermine war support? I think we do not want to know if Ukraine did it, or if other western agencies acting on Ukraine's behalf did it. Like I said, I am arguing against myself. I would cite comprehensive radar data storage if I though it, unicorns or magic dragons existed. Oh look, scientific babble. Ignored and dismissed. As is your outlandish theory that data is scrubbed and analysed before storage. By whom. What project budget number are you citing to finance this? If radar data had been systematically stored, then it would have been used in ship inquiries. Each local radar station purges its caches to avoid data failure. 25 mb/s adds up fast. I see that you have never actually seen a radar monitor. There are literally 100ds of contacts per sweep. More if the weather is poor.
jergul large member	Thu Nov 17 10:18:43 Feel free to tell us how machine learning at each radar station is helping collect all this data in a compact way. Why not evolve your fairytale?
Seb Member	Thu Nov 17 15:53:31 jergul: I'm suggesting that police and normal law enforcement agencies would not have access to military radar data. It's being pursued as a national security matter, not a law enforcement operation. We know that because pretty much all three countries have said they are not going to cooperate and disclose national intelligence - which almost certainly means that law enforcement agencies are not being disclosed the data internally either. " Because nobody want politicians to have to chose to not escalate?" Release of classified information is normally in the gift of political bodies. So I would characterise it that govt agencies - rather than independent law enforcement bodies - are not going to declassify information and pass it to law enforcement agencies if it is going to cause problems for them. That is the point of classification regimes in the first place. I'm surprised you find this surprising. "I think we do not want to know if Ukraine did it, or if other western agencies acting on Ukraine's behalf did it." Which would require exactly the same thing to be happening that you find so surprising. "I would cite comprehensive radar data storage" You know as well as I do that storing the contacts etc. for years would fit on your laptops hard disk. Pretending they would need to capture the time series data for each sensor is illiterate. "Oh look, scientific babble. Ignored and dismissed." Well yes, this is called sticking your head firmly up your arse. "25 mb/s adds up fast" 25mb/s ignored and dismissed. You would store the analysed, not the signal return. ". There are literally 100ds of contacts per sweep. More if the weather is poor" Yeah. But I guess you haven't thought that each one of those contacts is four numbers. Even floating point 1000 contacts per second can be specified with say about 7000 thousand oh, lets go crazy and say 32 bit floating point numbers a second (range, bearing, azimuthal angle, velocities from doppler and cross-section return). That's 225 kb per second. So under 1 terabyte of data a year per radar station, assuming they keep the sampling at 1 second intervals. Which they probably wouldn't as it's far higher than needed. It's peanuts in storage terms. Of course they keep it. "By whom" John, he's this guy they get to eyeball the signal traces that come in off the channels. They come in on a thousand old style graph plotter machines - he quickly measures the signal amplitudes and then does the maths on an abacus to work out the range and baring of the contacts based on those analogue signals, then he runs over to a map and draws them down so the radar operators can see where they are. John is getting on a bit, his hands are tired and old, but until we figure out how to automate this stuff with - oh - a mix of analogue and then digital signal processing; we just have to rely on John and his abacus and 25MB/s raw signals from the radar receiver channels.
Nimatzo iChihuaha	Thu Nov 17 16:11:47 Ehum I found a paper 2.5 The radar systems The three radar systems that required compatibility with the software was the Swedish Armed Forces Surveillance radar 640, Surveillance radar 861 and Radar system 871. The 640 system is a coastal radar used for detecting ships but can also detect low flying aircraft. It can detect targets up to a range of 50 km and has a angular velocity of 12-24 RPM adjusted by the operator [9]. This system also produces the shortest messages of 18 bytes. The 871 radar is a low height radar with a range of 100 km. It is mounted near the coast to detect threats both in the air and on water and produces messages of between 48 and 50 bytes[8]. Among the three radars, the 861 system has the longest range of up to 300 km and it also registers height of an encountered target. The target reports of the 861 has a length of 34 bytes [7]. Both the 861 and 871 system has an angular velocity of 6 RPM . http://www...t/diva2:1225412/FULLTEXT01.pdf
jergul large member	Thu Nov 17 19:10:24 Nimi That was a bit random. Seb You are truly a CT nutter.
jergul large member	Thu Nov 17 19:20:37 Nimi Ah, I see. Message bytes per contact per rotation. Nothing that contradicts what I said. Here is the code for a single contact in one sweep of the radar: Frame 2; Length: 48 3 4 Block 2 5 CAT 048 6 LEN 48 7 8 1. Record of Block 2 (2. Record absolute) 9 10 I048/010[SAC: 099; SIC: 100] 11 I048/020[TYP: 1 (Single PSR); ACT; RDP1; DEF; DEF] 12 I048/040[RangeCentroid: 13.405 km; 13 AzimuthCentroid: 84.858 deg] 14 I048/130[PRI Azimuth extent: 1.934 deg; 15 PRI Max Magnitude: 39.0 dBm] 16 I048/SP [Range extent: 0.05 meter; 17 Signal to Noise ratio: 11.50 dB; 18 Blip to Scan ratio: 0; 19 Average Magnitude: 35.50 dB; 20 RadarCrossSection: -2.409 dBsm; 21 No. of CPIs with Async.Interf.: 0; 22 No. of hits with Async.Interf.: 0; 23 Scan mode: 1 (Normal); 24 Number of Hits in High Beam: 9; 25 Number of Hits in Low Beam : 0; 26 Best estimated Radial Velocity: -0.10 m/s; 27 Second best estimated Radial Velocity: -0.10 m/s; 28 Plot in Notch Filter (1-8) : 0 0 0 0 0 0 0 0; 29 Plot in Notch Filter (9-16) : 0 0 0 0 0 0 0 0; 30 Plot in Notch Filter (17-24) : 0 0 0 0 0 0 0 0; 31 Plot in Notch Filter (25-30) : 0 0 0 0 0 0] 32 33 f3 01 01 04 00 00 00 00 00 20 07 3d 3c 58 18 2c 34 27 1c ff f8 00 02 17 00 00 47 f6 97 00 00 00 00 35 01 00 09 00 00 ff ff ff ff 00 00 00 00 Point is. Understand what you are posting ;)
jergul large member	Thu Nov 17 19:31:20 But fair enough. For a typical military grade radar, you would have something in the region of 1 Mb/s. I was quite far off. I can live with that :D. There are after all 3600 seconds in an hour.
jergul large member	Thu Nov 17 19:35:16 In the next chapter of this saga, lets here Seb tell us how all night vision googles in Nato are collecting all data at all times for security reasons.
Seb Member	Fri Nov 18 03:18:37 Nim: "he target reports of the 861 has a length of 34 bytes" That sounds about right my crude estimate was around 28 bytes per contact. Jergul: You have completely misunderstood Nim's post. 34 bytes per contact per second > so lets say 10,000 contacts per second > 340kb/s. 340 * 60 * 60 * 24 = under 30 gigabytes aday c 30 gigabytes a day. 5 Terabytes of data for 6 months. That wouldn't quite fit on my desktop - but a commercial storage rates its about 500 dollars a month. Wherever will we find the finance for such a program That would not quite fit on my desktop. You are off by a factor of around 10,000.
jergul large member	Fri Nov 18 03:26:52 I checked. The code is 125 kilobytes, not 34 bytes. It is after all 35 lines long.
jergul large member	Fri Nov 18 03:35:54 "ffective and accountable radars are important as the surveillance of a country’s border is of high-priority. To accomplish this all radars used for civilian or military applications needs testing and maintenance to avoid col- lecting incorrect data. This is done with real-time analysis, but also by analyzing collected data at a later instant. When the data is collected it is saved as a binary file. Saab has struggled to find the proper tool for ana- lyzing the stored data and have used unorthodox methods such as capturing screenshots and browsing through the data point by point. The method used for capturing the screenshots was only available for one of the radar systems which made the analyzis even more restricted. For these reasons an analytical tool needed to be developed"
jergul large member	Fri Nov 18 04:12:55 https://gyazo.com/7cf15fa581202ce7e1f8bef04aad0621 125 bytes per second x 10 000 contacts per second = 1,25 mb/s = 108 gb/day But the actual point here is that Nimi documented the producer and upgrader of the radar types in question does not have method for analysing data except by screenshots of the radar screen computer in action. This is where I got my number from. I assumed something like that was happening to the extent data was being stored at all. So we can lay to rest the theory that historical radar data is available. The producer does not have the tools to do that.
Seb Member	Fri Nov 18 04:38:36 Jergul: You realise that you can generate that message from a far smaller data payload?
Seb Member	Fri Nov 18 04:45:26 "Figure 4 is an example of a category 048 messagge produced by the 871 radar system after translation." Most of that text message is generated from understanding what each byte of the binary message means. The system isn't sending an ASCII file with loads of redundant human readable text.
Seb Member	Fri Nov 18 04:47:59 Only jergul would take a paper showing someone literally implementing a feature set he says is impossible on data sets he says don't exist as somehow proving his point.
Seb Member	Fri Nov 18 04:54:18 Bottom line, you could store 10,000 contacts a second per radar receiver and it would fit on generic consumer grade storage devices you can fit in an unmodified desktop computer computer chasis.
Seb Member	Fri Nov 18 04:58:20 "But the actual point here is that Nimi documented the producer and upgrader of the radar types in question does not have method for analysing data except by screenshots of the radar screen computer in action." Yeah, but it's quite normal for multiple parties to be involved. Likely a different contractor responsible for integration. SaaB hadn't got around to building a decent viewer for helping to use historic data to test their performance, so they paid a university to build one and it was an easy internal project. That's the message here. Not "this is some crazy high end shit that national security agencies almost certainly don't have because the data is crazy big and expensive".
Seb Member	Fri Nov 18 05:14:38 Lines 33,34,35 - that's the raw binary target report (in this case 45 bytes) You can generate everything in the ASCII formatted message from that block of bytes.
Seb Member	Fri Nov 18 05:35:34 "125 bytes per second x 10 000 contacts per second = 1,25 mb/s = 108 gb/day" You just said it was 125 kB (you are wrong) but that means you think 108 TB a day. Does that sound plausible from a bandwidth perspective for a radar receiver to talk to its control room? But let's go with 108gb of data per day per radar. A network of 30 odd radar station would need 500TB storage for six months data. AWS will give you high frequency (you don't need this) access for over 500TB at 2.1 cents per GB per month. So anual cost of $11,000. Instant access archive is 20% of that. Can you not just accept you are wrong on this point - there's nothing extraordinary about expecting this to be stored?
Seb Member	Fri Nov 18 05:40:31 Even if you thought it was 100TB a day, you are talking a paltry 10 million USD for a six month archive for a radar network of 30 stations. (I picked 30 as I recall that being what Denmark is refreshing).
Nimatzo iChihuaha	Fri Nov 18 05:56:10 Honestly, strange as it is, I became curious. https://opendata-download-radar.smhi.se/explore/# Here is the swedish metreology institute, they have an open access radar archive that goes back to 2008. I took a random day in 2008 and zip fil is 22 mb. Took two more recent days and they were 6 and 10 mb. Looks like radar files are quite small?
jergul large member	Fri Nov 18 06:30:45 Seb I took the lines and had them counted. 125 bytes. We have established that the producer and contracted upgrade company SAAB has to use screenshots to look at historical data. It also documents that the way data is stored is not a secret. Why would it be? Nimi Depends on how many saving you are saving. We have established that it is 125 bytes per contact. What you are looking at has 5 minute intervals.
jergul large member	Fri Nov 18 06:45:55 Seb Obviously just a typo. 108 gb is correct like I said. What is with the cut-off at 6 months? Why not 6 years or 60 years? My point was always that this would be a major project that would be publicly known. It would also have been used as evidence in ship inquiries. Factually, it is what I said. Data is stored until caches are purged unless Saab has requested that the operators take screenshots and send them over and png files. I would frankly not even assume the radar processing units are connected to the internet. That seems like a huge security risk.
Seb Member	Fri Nov 18 07:09:22 Nim: They are - that's the output messages as binary. jergul: "Data Item 048/040, which represents thevpolar coordinates of the target, are stored in byte 11 through 14 for the radar system 640, and 13 through 16 for the systems 861 and 871" ... "Figure 4 is an example of a category 048 message produced by the 871 radar system after translation. On row 13 the 048/040 Data Item is displayed." You have no need to count. They say it explicitly, and you can see the raw data byte on the bottom of the translated message - each hexadecimal pair is a byte of data. You generate the text simply by understanding what each byte in the binary file means according to the ASTERIX protocol. The first three bytes of any stream tell you everything you need to know to decode the entire message. I don't know how you are getting to 125bytes by counting lines, but you have totally misunderstood the paper if that is what you are doing. Target report is less than 50 bytes. But even if you were correct and and it was 125 bytes, the storage for an entire networks data for six months assuming up to 10,000 targets per radar per second would cost about 10,000 USD. It is trivial. "It also documents that the way data is stored is not a secret. Why would it be?" I never said it was. But if a different contractor, say an integrator had built the systems used by the people actually monitoring the radar, then it's entirely possible SAAB had never built their own (or at least not for consuming the ASTERIX protocol outputs) - it was always easier for a team to do weird ad-hoc stuff in the moment rather than take one of their engineers off doing whatever it was they were doing to build their own. Until they realised it was dumb and threw some cash at a university to make them one - and yes its trivial. Which is my point. You are using a paper that shows it is trivial AND ACTUALLY DONE (you don't pay a university to build a viewer for a data set that does not exist) to somehow try and somehow prove that it is implausible that it can be done. And you are also not even reading the paper and insisting that when the paper explicitly states the length of the target reports in bytes, doing some mad ad-hoc line counting of an ascii rendering to try and infer the size of the message, when it is right there in front of you in black and white.
Seb Member	Fri Nov 18 07:18:50 "What is with the cut-off at 6 months? Why not 6 years or 60 years? My point was always that this would be a major project that would be publicly known." What operational usage would be of second by second data from 60 years ago? "I would frankly not even assume the radar processing units are connected to the internet. That seems like a huge security risk." Depends what you mean by the internet. e.g. this kind of stuff in the UK would go through PSN and new equivalent - it's not a dedicated infrastructure, its an overlay on the civil networks. http://en.wikipedia.org/wiki/Public_Services_Network Yes, it is publicly known though - the maritime surveillance networks procurements are a matter of public record. You are just assuming that a requirement to store data would be so big as to command its own programme. Which is nonsense. It's tiny.
Seb Member	Fri Nov 18 07:21:06 ". Data is stored until caches are purged unless Saab has requested that the operators take screenshots and send them over and png files." The paper you are citing from is literally describing a pre-existing messaging protocol (ASTRIX) to allow individual radar stations in a network to share data with each other - and described its application to historic data. If you think the data just sits in a control room of a physically integrated radar station, what is the point of this protocol? Where does it come from? Where does the raw binary file data that the viewer is viewing come from?
Seb Member	Fri Nov 18 07:22:44 https://www.eurocontrol.int/asterix
Seb Member	Fri Nov 18 07:27:03 https://www.isasi.org/Documents/library/technical-papers/2011/Schuurman/PAPER-ISASI-FARRELL-SCHUURMAN-FINAL-20110627.docx "The recording of radar data is detailed by the International Civil Aviation Organization (ICAO) in ANNEX 11. Historically it was recommended to record surveillance data from primary and secondary radar equipment. Furthermore it was recommended that recordings should be retained for a minimum 14 days. ICAO has specified in a State Letter (AN 13/13/1-05/37) that “Radar Recording of primary and secondary surveillance data is no longer recommended but mandated” as of 23rd November 2005. “Automatic recordings shall be retained for a period of at least thirty days. When the recordings are pertinent to accident and incident investigations, the recordings shall be retained for longer periods until it is evident that they will no longer be required”. So one can conclude that an international protocol is in place for the preservation of radar data for accident investigation purposes, though implementation of this protocol is a matter for local regulation. "
Seb Member	Fri Nov 18 07:28:10 Do we really need to go on and on with this? Of course national security organisations of the baltic coast states have access to historic radar tracks that could identify ships of interest going back months.
jergul large member	Fri Nov 18 09:18:00 The quote covers neither maritime, nor military radar use. A mandate is also something to force compliance, not document what is being done. 30 days is also way too short a window. We have documented that the radar producer could only access historical data in the form of screenshots. There is a reason you are not finding what you are trying to find.
Seb Member	Fri Nov 18 09:58:03 Jergul: Lol. Ok, so civil agencies are expected to hold up to 30 days of radar data and do so in Asterix binary files; but it would be unthinkably "extraordinary" for military / security to do something similar when part of the point is to support e.g. detection of piracy, smuggling or other nefarious activity; and indeed you are arguing that it's technically infeasible and disproportionately expensive to implement. Yeah, yeah. Of course. Come on, this is embarrassing.
Seb Member	Fri Nov 18 10:01:34 Jergul: "We have documented that the radar producer could only access historical data in the form of screenshots." No we haven't. Not at all. What we have documented is that the radar manufacturer hadn't built an app for viewing Asterix protocol binary outputs and used work arounds until the commissioned a university to do it (not even worth their time to do, it is so simple). "There is a reason you are not finding what you are trying to find." You are being literally buried up to your eyeballs in it! I'm going to stop now. You can stay on like a dismembered torso screaming "come back 'tis but a flesh wound" if you like.
Seb Member	Fri Nov 18 10:04:10 I do wonder sometimes jergul. I mean *I* know that you have enough technical knowledge to know you are wrong at this point. And you know that I know this. And I know that you know that I know this. So it's just silly.
jergul large member	Fri Nov 18 10:51:12 Seb Continue to search for a single example of stored maritime radar data being used by maritime courts (ship inquiries), for a non-existent standard for storing maritime data, and for the non-existent projects that keep such data stored long term. You can try looking at Battlefield Management Systems I suppose. Wavehawke or similar. Now just find the naval battlefield management system Sweden/Denmark uses and see if it has been purchased for its shorebased radars. (Hint. It has not) I am willing to entertain that radar data may be purged slightly less frequently than daily and that binary files can sometimes be moved from the radar system by way of thumb drives to internet connected machines.
jergul large member	Fri Nov 18 10:53:36 http://www...Command%20Information%20System
jergul large member	Fri Nov 18 11:01:04 ahaha, Saab makes the Swedish Battlefield Management System. You know, the company looking at screenshots from radar operators to when it needs to collect historical data.
Seb Member	Fri Nov 18 11:06:04 Jergul: I've not been searching and have no intention to start because it is a total red herring. The fact the link above refers to incident investigation is coincidental. It's abundantly clear that radar target reports are tiny amounts of data, they are routinely stored, there are whole industry standards for their storage and distribution. And as we see, in civil aviation they are actually required to keep data available for 30 days, something you were arguing was technically or financially infeasibile. You can keep arguing it is extraordinary to think so. But even your projection of storage requirements agree the storage costs for an entire network of such radars is unlikely to exceed 10k USD a year. So yeah, of course the military archive radar data. For much the same reason civil aviation is required to do so. They are likely to keep it longer because civil aviation accidents come to light quickly, but security incidents might not emerge immediately and need to be investigated retrospectively.
Seb Member	Fri Nov 18 11:07:50 "The system has a number of interface ports (including two RS232 serial ports) that can be configured to accept information from the ship's sub-system, normally in the form of track information from radar, sonar" So proprietary raw signal data and probably not Asterix protocol, right?
jergul large member	Fri Nov 18 13:25:23 Asterix is the standard according to Nimi's source. A battle management system (BMS) would certainly accept Asterix. Note the singular lack of internet access. A vessel's BMS is in principle self-contained for system integrity reasons. I am arguing that such projects are so complex that you would be able to tell me the name of the project and the management program used to store data. Find what the Swedes are using for their coastal radars so we can look at the specifications. Dont expect much. Saab was getting screenshots taken by radar operators when it wanted to collect data to analyse. This suggest a singular lack of sophistication and cache purges with great regularity. Perhaps not daily. But close. Otherwise, it is at it always has been. Data is stored locally until the caches are purged.
jergul large member	Fri Nov 18 13:28:42 The baseline here is close to daily cache purges and data transfers using thumb drives only when specifically requested. Find evidence of greater sophistication.
Seb Member	Fri Nov 18 13:39:11 jergul: A standard for information exchange across agencies - not for integrated systems which will be done natively. A battle management system would likely not use asterix as you would get higher bandwidth by reducing the bytestream to exactly the information needed (you are hard integrated to a specific sensor so there is a bunch of stuff you would not need in an asterix message). "I am arguing that such projects are so complex that you would be able to tell me the name of the project" Why is it complex? It would be equally complex for each ATC system to do the same, do they have sepcial named projects? It'll be part of the programme for building and operating the coastal radar systems in the first place. "This suggest a singular lack of sophistication and cache purges with great regularity." Lol. No. Not at all. It implies that getting data out of govt system is a bit of a nightmare for burea.cratic reasons,
jergul large member	Fri Nov 18 13:44:31 Seb Find the name of the battlemanagement system the Swedes are using for their radar systems (it is a system after all). If you cannot find one, then assume it is propriatary SAAB. Which means screenshots transferred to thumbdrives by request only before being plugged into the internet and transferred to Saab for maintenance and analysis purposes.
Seb Member	Fri Nov 18 16:46:03 "Which means screenshots transferred to thumbdrives" Lol no. Photos of the screen. Thumbdrives. Lol. No just no.
jergul large member	Fri Nov 18 18:02:55 Seb Its a closed system. Thumb drives are the easiest way to get it from the closed system to the internet. The screen shots are of course documented by Nimi's link.
Seb Member	Sat Nov 19 05:21:52 The restricted machines we used to use at il4 had their usb ports blocked in software. MoD laptops for il5 and higher have their USB ports physically disabled. The system you quote refers to old school serial ports (much faster of everything is done at the lowest possible layer). No. Just no. Connection to a secured network is much more secure than having a fucking USB port on your device. I'm 95% sure that what they were doing is literally photographing the screen. The next step up is getting Astrix binary files from a data centre where it's accessible by hardware that actually has an interface for physical media or enough permissions to FTP it out through some kind of staging area. SaaB probably doesn't have a software system for doing pause/rewind/play of Asterix files because normally they would just plug into their system. Getting real world data from live use is more complicated.
jergul large member	Sat Nov 19 05:57:34 Seb Yes, just yes. You are struggling with the concept of what a radar system is. So you are using elitist prick jargon to say exactly what I am saying. A secure hardwired network with "an interface for physical media" Ie a thumb drive. The system you quoted. It is not an old system. Far younger than most of the maritime radar systems Sweden is using. Saab does not have a system because it rarely is paid to collect and review radar data and does so only for maintenance purposes. I am 95% sure a conscript is definitely not snapping cellphone pictures of operations including his monitor. Not without wanting to be charged with espionage.
jergul large member	Sat Nov 19 06:01:07 "software blocked" lol, you mean usb slots were disabled in windows hardware settings and normal users not having admin privileges to enable them again I think :).
Seb Member	Sat Nov 19 08:27:35 No, nobody uses dedicated physical infrastructure like that anymore. It's unnecessary and extortionately expensive with little benefit. And no, you wouldn't take a screen grab from the control room and send it to a data centre for extraction. Why would you even have an option to take a screen grab in the first place rather than export the data file, if taking a snap shot of the current situation is an operational requirement? (Hint: it is, but it will go over the network to a data centre and is an utter pain to get data out of even not that highly protected data centres or managed clouds, which is why they probably resorted to snap shots). The system can probably use some proprietary protocol locally that's more bandwidth efficient (which is what the manufacturer will have viewers for) and export Astrix data for exchanges within the secure network. Nothing you are saying makes the slightest bit of sense if you bother to think about it for more than a second or two. "I am 95% sure a conscript is definitely not snapping cellphone pictures of operations including his monitor. Not without wanting to be charged with espionage." He isn't likely to be a conscript is he, he's likely a specialist contractor with appropriate clearance working with the operator to do service improvements. And rather than spending months and several thousand dollars getting a staging area set up (that's how the SIs get you - change requests) to get data extracted, it has been convenient to just take a photograph. Then a big piece of work comes up, or there's a network upgrade or some such that makes this viable and suddenly you can get hold of Asterix files instead. But you haven't bothered building an Asterix viewer because normally you use your propriety data format. You can get your team to build one, but that's a waste of time. Outsource it to a local university as part of your talent pipeline. "you mean usb slots were disabled in windows hardware settings and normal users not having admin privileges to enable them again I think :)." Yes. That's a long a laborious way of saying what I said (actually probably in bios, but thats a detail). Any particular reason you feel the need to summarise it? But that's the point IL4 is "embarrassing". Secret systems - things that are part of operations - they physically disable USBs. You don't want tsome tit sticking a thumb drive into an operational system ever. Conscripts are easily bribed. And your entire thesis is that the data never leaves the control centre over network because zomg so much data. Like I said, this is absolutely ridiculous conversation. There will be retained radar contact data available to investigators.. It's feasible. It's useful. It's not expensive.
Seb Member	Sat Nov 19 08:28:38 *Hint: it is, but it will be in the form of Asterix files that...
jergul large member	Sat Nov 19 08:40:18 Seb I can tell you have never been in the military or know much about how stuff at sea works (hint. It generally works poorly. It is incredibly conservative). You are just doing your thing. Babble away until people think you are an authority. Find some trivial thing you think might make sense to an imaginary audience and leverage it heavily. The Swedish maritime radar systems are closed systems. They most definitely have a hard wire cable splice to a secure machine that can download data to a thumb drive and upload patches from a thumb drive. We have already established that Saab views data in the form of screenshots. Prove me wrong. Find the Battlemanagement system the Swedes are using that is not proprietary Saab "hmmm, we will need some screenshots please" levels of sophistication.
jergul large member	Sat Nov 19 08:46:12 I do that too btw. Its an elitist prick move.
Nimatzo iChihuaha	Sat Nov 19 08:47:51 sigh... guys is this going to be a multi thread thing? I can call the roper authorities and ask how long radar recording need to be archived. Presumably nobody saves anything for any amount of time, unless there are good reasons, which normally equals to there being regulatory demands or requirements between institutes and authorities. Those documents would be another place to look at.
jergul large member	Sat Nov 19 09:30:57 Nimi Oh, most def. Its the ad nauseam fallacy. Kick at it until everyone throws up. For radars falling under civil aviation, it is 30 days. Seb found that. For maritime radars. No regulatory demands for data storage. In the same way mandated transponder use has huge gaping holes. The premise for freedom of navigation is actually just a huge fuck you to local authorities that might otherwise occassionally make life awkward for ship masters wanting to do their own thing, at their own pace, and with as much secrecy as they desire.
Nimatzo iChihuaha	Sat Nov 19 10:09:18 Jergul I agree that regardless of the size of data files, which seem to be small enough to be manageable unless there are requirements to save them then nobody is saving them to any reliable degree of fidelity and coverage. I guess the surprise here would be if indeed maritime (law enforcement) and the military wouldn't be required to do so when virtually all civil infrastructure has these types of storing requirements. 30 days seems reasonable. I guess stranger things have happened, but to enforce a surveillance regime on radar stations is not that difficult. Being part of critical civilian and military infrastructure they are either operated by the state or under strict regulation already. Not as complicated as enforcing compliance on every ship out on the sea.
Nimatzo iChihuaha	Sat Nov 19 10:25:32 I found this, the text implies that Jergul is correct. http://doc...v-svenska-fjarranalysdata.html "Både SMHI/FIMR och Kustbevakningen köper för närvarande data på kommersiella grunder för realtidstillämpningar och det vore sannolikt rationellt att tillhandahålla dessa data samordnat för att till exempel få ner priset. Ett annat alternativ är att i större utsträckning än idag skriva avtal med dataleverantörer direkt innan satelliterna skjuts upp för att få billigare data till nationella arkiv och operationella tillämpningar. Radardata sparas inte generellt av dataleverantörerna och därför finns inte centrala lager där data kan hämtas över ett godtyckligt område." Anyway the gist of it is that the Swedish coast guard buys their real time radar data and the suppliers do not generally save them there is no central archive to get them from. Elsewhere in the study (exploring the need for a central archive) it says the coast guard has little use for archived radar data.
Seb Member	Sat Nov 19 10:50:15 Jergul: "We have already established that Saab views data in the form of screenshots." We have proven the opposite, because they paid a university to make a viewer for archived data *because* they were using ropey screenshots. Why make a viewer for archive data that does not exist? Why have a function to export data in a format optimised for efficient real time stream to systems that are not configured to take raw data from a specific sensor? Nim: Reference to satellites makes it sound like synthetic aperture radar satelite imagery? Sweden operates a military hf coastal surveillance radar, so why would they buy that data? It makes more sense that is referring to satélite based look down radar. I would be genuinely astounded
jergul large member	Sat Nov 19 18:39:24 Seb You have not proven that the student BSc project was paid for by Saab or implemented by Saab. All we know is that Saab relied heavily on screen shots because that is the level of sophistication its properetary battlemanagement system for maritime radars has. You are astounded because you are a technocrat with a pretty sublime connection to the real world.
jergul large member	Sat Nov 19 18:41:40 Nimi My argument was always that bandwidth and storage requirements would suggest a project large enough to find a footprint of it in fact such a project existed. Seb disingeniously suggested I meant bandwidth and storage was out of reach. A silly red herring of course.
jergul large member	Sat Nov 19 18:45:59 The data of course exists at any given moment in time. Data shown on monitors has the Asterix format. It is even saved. For a while. Until caches are purged.
Seb Member	Sun Nov 20 04:18:18 "Analytics tool for radar data was a project that started when radar specialists at Saab needed to modernize their tools that analyzes binary encoded radar data." The project is explicitly to produce something for Saabs radar specialists. "All we know is that Saab relied heavily on screen shots" No we don't. The paper explicitly said this method was used for only one radar system. "Saab has struggled to find the proper tool for ana- lyzing the stored data and have used unorthodox methods such as capturing screenshots and browsing through the data point by point. The method used for capturing the screenshots was only available for one of the radar systems which made the analyzis even more restricted. For these reasons an analytical tool needed to be developed." "that is the level of sophistication its properetary battlemanagement system for maritime radars has." You are making that inference without basis. The operational data is encoded in a format they hadn't built a viewer for. That's all we can conclude. "argument was always that bandwidth and storage requirements would suggest a project large enough to find a footprint of it in fact such a project existed." Except your own sums, inflated at they are, show that the bandwidth requirements are minimal and six months of data for an entire network would fit in commercial grade drives that could be mounted in an unmodified standard desktop chasis.
Seb Member	Sun Nov 20 04:20:18 It definitely would not require it's own project. It would likely simply be part of the REMO project that SaaB was the integrator for. A firm called ITT provided the radars as a COTS offering. And this is likely why they didn't have a viewer for Astrix data. They are used to using their own proprietary data formats.
jergul large member	Sun Nov 20 07:27:51 Seb You are just incorrect. Sorry. This point is declared proven by this thread. "Why bother with the justification? I am sure I could find something if I wanted to. BS justifications is one of the elitist tricks done to make people with long post secondary educations feel they have not wasted their time." You just wasted a lot of time. Which is fine. That is what we do in this forum.
jergul large member	Sun Nov 20 07:35:43 That was not a walkback btw. I think the justifications was a waste of time. You made a claim. Swedish maritime data is stored and accessible. You failed to prove that claim or significantly substantiate that claim. I gave you a pretty low bar. Find the name of the battlemanagment system the Swedes are using or assume it is using propretary thing so unsophisticated, you are left gasping, no look, the student paper probably fixed that and this proves radar data is being stored forever now since as least 2018 when the student paper was written. A bit sad frankly. Not your finest moment.
	show deleted posts