Monthly Archives: November 2024

Coding

2024 Reading list (updated as we go)

Figures from the Global Carbon Budget 2024 : https://robbieandrew.github.io/GCB2024/

C Just In Time : cjit https://dyne.org/cjit/ : fast like hell

State of HTML (and related) https://2024.stateofhtml.com/en-US

Neural Networks : Zero to Hero https://karpathy.ai/zero-to-hero.html

state of swe jobs market https://newsletter.pragmaticengineer.com/p/state-of-eng-market-2024

reinforcement learning explained : https://ai.gopubby.com/how-did-alphago-beat-lee-sedol-1a160d76612b

O(1) lfu : http://dhruvbird.com/lfu.pdf

C11 atomics (atomic_int for ex) are still not supported by C++ when including C code : https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p0943r6.html and https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0063r0.html

Round Robin DNS : https://datatracker.ietf.org/doc/html/rfc8305

Srinivasa Ramanujan : https://www.quantamagazine.org/srinivasa-ramanujan-was-a-genius-math-is-still-catching-up-20241021/

What is legacy code ? According to https://understandlegacycode.com/blog/key-points-of-working-effectively-with-legacy-code/ “Legacy Code is code without tests”

[thoughts] we are transitioning from swe tools to product design tools; llms are blending the boundaries between code, UI/UX, and product ideation.

Are LLMs reasoning ? https://arxiv.org/pdf/2410.05229

Open-Meteo is an open-source weather API https://open-meteo.com/en/docs

Remote work is young and we have not built up methodologies or just even habits or practices : https://intenseminimalism.com/2024/the-myth-of-the-missing-remote-work-culture/

A life spent watching the sky : https://www.majakmikkelsen.com/film

Hard life for rust and linux : a proposal for a rust interface to fs .. https://www.youtube.com/watch?v=WiPp9YEBV0Q&t=67s

Segment anything : a new AI model from Meta AI that can “cut out” any object, in any image, with a single click https://segment-anything.com/

14 years since Go launched : the good and the bad by Rob Pike https://commandcenter.blogspot.com/2024/01/what-we-got-right-what-we-got-wrong.html

Writebook : everything you need to edit and publish your online books

Merchants of complexity : https://world.hey.com/dhh/merchants-of-complexity-4851301b ( on the attraction for complexity read here )

Tired of slack and not owning the data ? https://once.com/campfire#requirements

Something in between a Product Manager and a Software Engineer : Product Engineer i.e. PMs are sometimes not enough technical and SWEs are sometimes not enough product oriented https://refactoring.fm/p/how-to-become-a-product-engineer

myspace reborn https://spacehey.com/

Stephen Wolfram on neural nets : https://writings.stephenwolfram.com/2024/08/whats-really-going-on-in-machine-learning-some-minimal-models/

Some good recommendations https://levelup.gitconnected.com/follow-these-6-patterns-or-i-will-reject-your-pull-request-fc08f908e7fe :

  • Early return and align the happy path left
  • Avoid boolens in methods signature
  • Avoid double negations
  • Use default values to avoid unnecessary else in initializations
  • Avoid functions with side effects


3D Mesh generation with object imageshttps://omages.github.io/

Hetzner de servers auction https://www.hetzner.com/sb

Ransomware victims : https://www.ransomware.live/#/recent

Red and Blue teams in cybersecurity : https://anywhere.epam.com/en/blog/red-team-vs-blue-team

How google is using AI internally https://research.google/blog/ai-in-software-engineering-at-google-progress-and-the-path-ahead/

Protecting artists from gen ai : https://glaze.cs.uchicago.edu/what-is-glaze.html

configuring core dumps in linux/docker https://ddanilov.me/how-to-configure-core-dump-in-docker-container

dolt, a version controlled database mysql compatible https://github.com/dolthub/dolt

MS/DOS 4.01 is open source https://cloudblogs.microsoft.com/opensource/2024/04/25/open-sourcing-ms-dos-4-0/

Contents shortage for AI :

Meta, for instance, trained its new Llama 3 models with about 10 times more data and 100 times more compute than Llama 2. Amid a chip shortage, it used two 24,000 GPU clusters, with each chip running around the price of a luxury car. It employed so much data in its AI work, it considered buying the publishing house Simon & Schuster to find more. 

https://www.bigtechnology.com/p/are-llms-about-to-hit-a-wall

Stop doing cloud if not necessary (I’m saying this since years..) https://grski.pl/self-host

Redis forks (after the licence change) :
– redict : https://redict.io/ Drew DeVault + others?
– valkey : https://valkey.io/ backed by AWS, Google, Oracle, Ericsson, and Snap, with the Linux Foundation; more to come imo.

nginx new fork https://freenginx.org/ (others forks are openresty)

Too much hype about Devin : Debunking Devin: “First AI Software Engineer” Upwork lie exposed! https://www.youtube.com/watch?v=tNmgmwEtoWE

Matt Mullenweg buys Beeper (already owns Texts.com and Element (New Vector)) consolidating his position in Matrix.org based messaging services : https://techcrunch.com/2024/04/09/wordpress-com-owner-automattic-acquires-multi-service-messaging-app-beeper-for-125m/

golang fasthttp (replacement for standard net/http if you need “to handle thousands of small to medium requests per second and needs a consistent low millisecond response time”. “Currently fasthttp is successfully used by VertaMedia in a production serving up to 200K rps from more than 1.5M concurrent keep-alive connections per physical server.” https://github.com/valyala/fasthttp

Back to basics 🙂 Bloom filter https://en.wikipedia.org/wiki/Bloom_filter

1 billion row challenge : https://github.com/gunnarmorling/1brc

golang : alternative to cgo ? https://github.com/ebitengine/purego

Command line benchmark tool : https://github.com/sharkdp/hyperfine

New jpegli, jpeg-xl derived : https://giannirosato.com/blog/post/jpegli/

Edge CDN techniques : Shielding from fastly i.e. use a designated edge cache instead of origin https://docs.fastly.com/en/guides/shielding on a edge cache miss.

Apple car not interesting anymore : https://www.bloomberg.com/news/articles/2024-02-27/apple-cancels-work-on-electric-car-shifts-team-to-generative-ai

golang error handling the Uber way : https://github.com/uber-go/guide/blob/master/style.md#errors

nginx forking : Maxim Dounin annouces https://freenginx.org/en/ on the nginx forum https://forum.nginx.org/read.php?2,299130

Quad 9 free dns 9.9.9.9 : https://www.quad9.net/

UI testing the netflix way : https://netflixtechblog.com/introducing-safetest-a-novel-approach-to-front-end-testing-37f9f88c152d

Check it out : the new super-ide https://zed.dev/

Lex/Yacc today : https://langium.org/

Inside Stripe Engineering Culture, a series a posts : https://newsletter.pragmaticengineer.com/p/stripe

I find truly interesting the point around promoting a write culture (Execs/Directors in tech blog, SWEs on tech blogs/internal technical documents) : https://newsletter.pragmaticengineer.com/i/140970283/writing-culture
I’m a long-time believer that writing clarifies thinking more than talking and writing persists information, makes it searchable, talking does not. “Verba volant, scripta manent” as the Latins use to say. But this idea shifted into “just enough” documentation (which means it is not necessary) in SW engineering latest methodologies so it is interesting that a multi billion company like stripe is going totally against the tide.

Chat/Instant Messaging protocols comparison

Comparison Table

ProtocolDecentralizedEncryptionMain Use CaseExamples
XMPPYesOptionalFederated messagingejabberd, Prosody
MatrixYesYesDecentralized chatElement, Synapse
SignalNoYesSecure messagingSignal, WhatsApp
SIPNoOptionalMultimedia communicationAsterisk, Linphone
IRCNoNoCommunity channelsLibera Chat, EFnet
ActivityPubYesOptionalSocial networkingMastodon, Pleroma
WebRTCPeer-to-peerOptionalReal-time communicationVideo calls, games
ToxYesYesPeer-to-peer messagingqTox, µTox
Slack RTMNoNoTeam collaborationSlack
MTProtoNoYesSecure messagingTelegram
JingleYesOptionalReal-time multimedia (via XMPP)Conversations, Dino

Social media protocols comparison

Comparison of ActivityPub, AT Protocol, and Matrix Protocol (mainly a reminder for myself) “

FeatureActivityPubAT ProtocolMatrix Protocol
Primary Use CaseSocial networking (microblogging, content sharing)Social networking with algorithmic controlReal-time communication (chat, VoIP)
Commands/VerbsCreate, Update, Delete, Follow, Like, Announce (Repost)Create, Update, Delete, Follow, Like, Repost, Algorithmic ChoiceSend, Receive, Join Room, Invite, Leave
Data ModelActivity-Object Model in JSON-LDJSON with user-defined schemasJSON (events model for real-time updates)
Transport ProtocolHTTPS, with JSON-LDHTTPS, with JSONHTTP (REST) and WebSocket, with support for end-to-end encryption (E2EE)
Identity ManagementTied to server domain (e.g., @user@domain.com), uses WebFinger for discoveryPortable DIDs for decentralized identityTied to server domain but portable; user ID format is @user:domain.com
FederationFederated, allowing instances to share content and social connections across domainsFederated with content and algorithm controlFederated, with real-time, synchronized state across servers
InteroperabilityWidely interoperable with other ActivityPub-compliant platforms in the FediverseDesigned for custom app experiences, interoperability is in developmentSupports interoperability with other Matrix clients; bridges to other protocols (e.g., Slack, IRC)
End-to-End EncryptionNot native to protocol but possible with extensionsNot natively specifiedBuilt-in and widely supported, particularly in 1:1 and group chats
ModerationInstance-based moderation policies, customizable filters and blocksUser-level and instance-level moderation, customizable algorithmsRoom-level moderation, with granular permissions for room admins
Popular PlatformsMastodon, PeerTube, Pixelfed, WriteFreelyBluesky Social, upcoming decentralized appsElement (main Matrix client), Synapse (server), bridges for Slack, Discord, Telegram, etc.

Summary of Key Differences

  • ActivityPub is best suited for federated social networking, particularly for applications that prioritize openness and content sharing across platforms in the Fediverse. It uses an Activity-Object model with JSON-LD and supports instance-based identity.
  • AT Protocol focuses on user control over content algorithms and portable identities using DIDs, with a vision for interoperability in custom social applications. It is also designed for federated social networks but with more control over data portability and algorithmic transparency.
  • Matrix Protocol excels in real-time, federated communication, supporting secure, encrypted messaging with granular moderation capabilities. It’s heavily used for chat, VoIP, and collaborative tools, emphasizing interoperability with other platforms through bridges.