Aggregator

Our anonymous submitter relates a tale of simplification gone bad. As this nightmare unfolds, imagine the scenario of a new developer coming aboard at this company. Imagine being the one who has to explain this setup to said newcomer.

Imagine being the newcomer who inherits it.

David's job should have been an easy one. His company's sales data was stored in a database, and every day the reporting system would query a SQL view to get the numbers for the daily key performance indicators (KPIs). Until the company's CTO, who was proudly self-taught, decided that SQL views are hard to maintain, and the system should get the data from one of those new-fangled APIs instead.

But how does one call an API? The reporting system didn't have that option, so the logical choice was Azure Data Factory to call the API, then output the data to a file that the reporting system could read. The only issue was that nobody on the team spoke Azure Data Factory, or for that matter SQL. But no problem, one of David's colleagues assured, they could do all the work in the best and most multifunctional language ever: C#.

But you can't just write C# in a data factory directly, that would be silly. What you can do is have the data factory pipeline call an Azure function, which calls a DLL that contains the bytecode from C#. Oh, and a scheduler outside of the data factory to run the pipeline. To read multiple tables, the pipeline calls a separate function for each table. Each function would be based on a separate source project in C#, with 3 classes each for the HTTP header, content, and response; and a separate factory class for each of the actual classes.

After all, each table had a different set of columns, so you can't just re-use classes for that.

There was one little issue: the reporting system required an XML file, whereas the API would export data in JSON. It would be silly to expect a data factory, of all things, to convert this. So the CTO's solution was to have another C# program (in a DLL called by a function from a pipeline from an external scheduler) that reads the JSON document saved by the earlier program, uses foreach to go over each element, then saves the result as XML. A distinct program for each table, of course, requiring distinct classes for header, content, response, and factories thereof.

Now here's the genius part: to the C# class representing the output data, David's colleague decided to attach one different object for each input table required. The data class would use reflection to iterate over the attached objects, and for each object, use a big switch block to decide which source file to read. This allows the data class to perform joins and calculations before saving to XML.

To make testing easier, each calculation would be a separate function call. For example, calculating a customer's age was a function taking struct CustomerWithBirthDate as input, use a foreach loop to copy all the data except replacing one field, and return a CustomerWithAge struct to pass to the next function. The code performed a bit slowly, but that was an issue for a later year.

So basically, the scheduler calls the data factory, which calls a set of Azure functions, which call a C# function, which calls a set of factory classes to call the API and write the data to a text file. Then, the second scheduler calls a data factory, which calls Azure functions, which call C#, which calls reflection to check attachment classes, which read the text files, then call a series of functions for each join or calculation, then call another set of factory classes to write the data to an XML file, then call the reporting system to update.

Easy as pie, right? So where David's job could have been maintaining a couple hundred lines of SQL views, he instead inherited some 50,000 lines of heavily-duplicated C# code, where adding a new table to the process would easily take a month.

Or as the song goes, Somebody Told Me the User Provider should use an Adaptor to Proxy the Query Factory Builder ...

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

This station completely transformed the town it now resides in

At a routine follow-up, her specialist was shocked - not just by her weight loss, but by a medical marker that had never normalised before. 'We were hugging,' Bernadette says.

A Stanway Villa U15 footballer was saved after collapsing mid-match, thanks to quick CPR from a referee and nurse.

The local council has been asked to build more public outdoor gyms as

You gotta start somewhere, and in this case astroboffins would have been nowhere without help from intrepid volunteers

NASA has published new analysis of its 2022 planetary defense test that suggests the mission slowed down the target asteroids, albeit infinitesimally.…

Colliding black holes were detected through spacetime ripples for the first time in 2015 by the Laser Interferometer Gravitational-Wave Observatory (LIGO), notes Space.com: Since then, LIGO and its partner gravitational wave detectors Virgo in Italy and KAGRA (Kamioka Gravitational Wave Detector) in Japan have detected a multitude of gravitational waves from colliding black holes, merging neutron stars, and even the odd "mixed merger" between a black hole and a neutron star... During the first three observing runs of LIGO, Virgo and KAGRA, scientists had only "heard" 90 potential gravitational wave sources. But now they've published new data from the LIGO-Virgo-KAGRA (LVK) Collaboration that includes 128 more gravitatational wave sources — some incredibly distant: [Gravitational-Wave Transient Catalog-4.0, or GWTC-4] was collected during the fourth observational run of these gravitational wave detectors, which was conducted between May 2023 and Jan. 2024... Excitingly, GWTC-4 could technically have been even larger, as around 170 other gravitational wave detections made by LIGO, Virgo and KAGRA haven't yet made their way into the catalog. One aspect of GWTC-4 that really stands out is the variety of events that created these signals. Within this catalog are gravitational waves from mergers between the heaviest black hole binaries yet, each about 130 times as massive as the sun, lopsided mergers between black holes with seriously mismatched masses, and black holes that are spinning at incredible speeds of around 40% the speed of light. In these cases, scientists think the extreme characteristics of the black holes involved in these mergers are the result of prior collisions, providing evidence of merger chains that explain how some black holes grow to masses billions of times that of the sun... GWTC-4 also includes two new mixed mergers involving black holes and neutron stars. [LVK member Daniel Williams, of the University of Glasgow in the U.K., said in their statement] "We are really pushing the edges, and are seeing things that are more massive, spinning faster, and are more astrophysically interesting and unusual." The catalog also demonstrates just how sensitive the LVK detectors have become. Some of the neutron star mergers occurred up to 1 billion light-years away, while some of the black hole mergers occurred up to 10 billion light-years away. Einstein's theory of general relativity can be tested with these detections, and "So far, the theory is passing all our tests," says LVK member Aaron Zimmerman, of the University of Texas at Austin. "But we're also learning that we have to make even more accurate predictions to keep up with all the data the universe is giving us." And LVK member Rachel Gray, a lecturer at the University of Glasgow, says "every merging black hole gives us a measurement of the Hubble constant, and by combining all of the gravitational wave sources together, we can vastly improve how accurate this measurement is." In short, says LVK member Lucy Thomas of the California Institute of Technology (Caltech), "Each new gravitational-wave detection allows us to unlock another piece of the universe's puzzle in ways we couldn't just a decade ago."

Read more of this story at Slashdot.

Steve Bassi, 71, said the theft was 'beyond the pale' and claimed police took more than an hour to arrive afterwards, though other uniformed officers came to order drinks in the meantime.

The Soham murderer became one of Britain's most infamous criminals after murdering ten-year-old schoolgirls Holly Wells and Jessica Chapman in 2002.

A video of the incident which occurred in Belgium on Christmas Day in 2020 has since been viewed tens of millions of times globally - but the culprit's identity has always remained secret.

Just 64 per cent of civil servants said their workload was 'acceptable', with more than one in four saying they don't have a 'good' work-life balance.

Gaffe­-prone David Lammy has done it again. For more than a year, he was Foreign Secretary, ­Britain's diplomat in chief on the world stage.

More than 100 victims of Steven Long suffered 'profound harm' after transferring money to his Universal Wealth Preservation fund.

According to policy documents on the party's website, it wishes to introduce more red tape for the owners of Britain's 13.5 million dogs and make parks return their animals to the wild.

The newly elected MP for Gorton and Denton had just spoken at the launch of Greater Manchester's Together Alliance - a group 'against the far Right' - when the incident occurred.

A Bridgerton star has reignited the controversy over whether George III's wife Queen Charlotte, who is depicted in the Regency-era drama, was black in real life.

Some people are easily bored. Paapa Essiedu, it appears, really can't be bothered with sequels. He stars in one series, then gets himself violently written out at the start of the next.

East Riding of Yorkshire Council said the sculpture would be dangerous because it would distract drivers and could be used as a 'take-off ramp' for daredevil drivers.

LibreOffice steward pleased says Commish saw the light, no longer requires Excel feedback form

UPDATED  The Document Foundation has taken a swipe at the European Commission over its consultation on guidance for the EU's Cyber Resilience Act – because the feedback template is only available as a Microsoft Excel spreadsheet.…

The UAE foreign ministry released the dramatic footage showing Iranian unmanned aerial vehicles (UAVs) being lined-up in an anti-missile system's crosshairs before being shot down.