The Zen of Python states "There should be one-- and preferably only one --obvious way to do it." OpenTelemetry is the obvious choice for traces but bad news for Pythonistas when it comes to metrics because both Prometheus and OpenTelemetry offer compelling choices. Let's look at all of the ways you can tie metrics and traces together with exemplars whether you're working with OTel metrics, Prom metrics, Prom-turned-OTel metrics, or OTel-turned-Prom metrics!
There’s a reason “i test in prod” isn’t a cheeky take but a lived reality. And that reason is there is no place like production. Not local dev or staging or other environment. This became clear when I deployed a tiny config change that passed all checks, reviews and pre-production environments that triggered a SEV-1.
Examining each step of the journey from PR to production I uncovered the snafu that had occurred (unsurprisingly it relates to overwriting key blocks on nested YAML files).
I’ll share how difficult it was to reconstruct the chain of events in the system compared to the ideal case of a highly observable system and how to share your own incident learnings since we all test in prod!
The misunderstanding that devops is a tool or team or role has tragically increased the gulf between developers and operators.If you have been unofficially on-call 24/7, toiled away in turmoil, managed initiatives across multiple silos sans project manager with a directive to “influence without authority”...surprise you have lived the contradiction that is feral devops!
Consequences of letting this fester are: burn out, learned helplessness and missing out on the speed that happens when developers and operators communicate openly. Taming feral devops is a journey you can start by gathering feedback on the status quo, iterating on onboarding and training and holistically examining on-call health.Whether you’re an individual contributor, manager or leader you will find ideas to experiment with to “domesticate” devops in your organization
SRECon23 Cognitive Apprenticeship in Action_ Alert Triage Hour of PowerPaige Cruz
Cognitive apprenticeship is the philosophy that it is more effective to learn in context and real-world situations compared to following a tutorial in a sandboxed environment. In a nutshell it is “learning-through-guided-experience” and shines when teaching problem-solving processes experts use to handle complex tasks like say…investigating an alert by spelunking in production observability and monitoring data. Learn how Alert Triage Hour of Power became a can’t miss meeting of camaraderie and system surprises!
Pushing Observability Uphill - The Single “Pain” of GlassPaige Cruz
For years tech companies have chased the fabled “single pane of glass” , the one observability tool to understand your system from north to south and east to west. Leafing through promo materials promising instant insights and seamless turnkey integrations you’d think increasing system observability is as easy as assembling a Lego set.
In my experience chasing the “single pane of glass” translates to “pain in the ass”. Survey data supports this revealing the majority of engineers cite tool sprawl as a minor or non-existent problem despite relying on several tools. As alluring as the siren call of “single pane of glass” is, let's be practical and examine how to best observe systems across a myriad of tooling.
From the telemetry buffet of metrics, events, traces and logs learn when to reach for which type and ways to bridge the gaps with links and enhance with context to free yourself from the fool’s errand of a “single pane of glass”.
Curious about containers? There’s a new generation of containers on the scene, Podman! Supporting secure, rootless containers for Kubernetes microservices, it was designed and built with the cloud in mind. Benefitting from the lessons learned out in the open from Docker, this next generation of containers will quickly become a trusted daily driver in your dev workflow.
Covering what you need to know as an end-user from the UI to the backend, sharing a real world use case leveraging Podman for open source observability workshops https://o11y-workshops.gitlab.io. Paige will share how Podman and the adorable seal mascots Caitlín, Maighréad and Róisín have transformed her local development!
This document provides an introduction and overview of instrumentation using OpenTelemetry. It begins with defining key concepts of observability and distributed tracing. It then discusses OpenTelemetry's SDKs, exporters and collectors for instrumenting code and exporting telemetry data. The document demonstrates both automatic and manual instrumentation techniques in OpenTelemetry and shows how to visualize trace data in Jaeger. It concludes by mentioning OpenTelemetry community resources for further learning.
From Cardinal(ity) Sins to Cost-Efficient Metrics AggregationPaige Cruz
Are you collecting just about every metric under the sun and the kitchen sink too? Understanding the cost of collecting metrics and the usefulness of those metrics is the only way to scale in a cloud native world. You can’t get away with just collecting everything as you grow. How can you make decisions about what to collect, what to drop, what to aggregate while still being able to alert, triage, remediate?
Gain immediate insights into high cost data (DPPS), when to drop time series data, and how to determine when the value of that data is at its lowest.
Distributed tracing is still finding its footing in many organizations today, one challenge to overcome is the data volume - keeping 100% of your traces is expensive and unnecessary. Enter sampling - head vs tail how do you decide? Let’s review the tradeoffs associated with different types of sampling strategies and how they can be mixed and matched.
This document discusses the evolution of observability from network/device monitoring (Wave 1) to application performance monitoring (Wave 2) to the current focus on observability (Wave 3). It notes that observability moves past proprietary data formats and interdependence of early solutions by enabling instrumentation once and observation anywhere. The document advocates for teams to own end-to-end observability of their services and dependencies, and notes that observability data can help teams understand where to invest engineering time to most directly impact customers and business goals.
Data Protection in a Connected World: Sovereignty and Cyber Securityanupriti
Delve into the critical intersection of data sovereignty and cyber security in this presentation. Explore unconventional cyber threat vectors and strategies to safeguard data integrity and sovereignty in an increasingly interconnected world. Gain insights into emerging threats and proactive defense measures essential for modern digital ecosystems.
Details of description part II: Describing images in practice - Tech Forum 2024BookNet Canada
This presentation explores the practical application of image description techniques. Familiar guidelines will be demonstrated in practice, and descriptions will be developed “live”! If you have learned a lot about the theory of image description techniques but want to feel more confident putting them into practice, this is the presentation for you. There will be useful, actionable information for everyone, whether you are working with authors, colleagues, alone, or leveraging AI as a collaborator.
Link to presentation recording and transcript: https://bnctechforum.ca/sessions/details-of-description-part-ii-describing-images-in-practice/
Presented by BookNet Canada on June 25, 2024, with support from the Department of Canadian Heritage.
AI_dev Europe 2024 - From OpenAI to Opensource AIRaphaël Semeteys
Navigating Between Commercial Ownership and Collaborative Openness
This presentation explores the evolution of generative AI, highlighting the trajectories of various models such as GPT-4, and examining the dynamics between commercial interests and the ethics of open collaboration. We offer an in-depth analysis of the levels of openness of different language models, assessing various components and aspects, and exploring how the (de)centralization of computing power and technology could shape the future of AI research and development. Additionally, we explore concrete examples like LLaMA and its descendants, as well as other open and collaborative projects, which illustrate the diversity and creativity in the field, while navigating the complex waters of intellectual property and licensing.
Video traffic on the Internet is constantly growing; networked multimedia applications consume a predominant share of the available Internet bandwidth. A major technical breakthrough and enabler in multimedia systems research and of industrial networked multimedia services certainly was the HTTP Adaptive Streaming (HAS) technique. This resulted in the standardization of MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) which, together with HTTP Live Streaming (HLS), is widely used for multimedia delivery in today’s networks. Existing challenges in multimedia systems research deal with the trade-off between (i) the ever-increasing content complexity, (ii) various requirements with respect to time (most importantly, latency), and (iii) quality of experience (QoE). Optimizing towards one aspect usually negatively impacts at least one of the other two aspects if not both. This situation sets the stage for our research work in the ATHENA Christian Doppler (CD) Laboratory (Adaptive Streaming over HTTP and Emerging Networked Multimedia Services; https://athena.itec.aau.at/), jointly funded by public sources and industry. In this talk, we will present selected novel approaches and research results of the first year of the ATHENA CD Lab’s operation. We will highlight HAS-related research on (i) multimedia content provisioning (machine learning for video encoding); (ii) multimedia content delivery (support of edge processing and virtualized network functions for video networking); (iii) multimedia content consumption and end-to-end aspects (player-triggered segment retransmissions to improve video playout quality); and (iv) novel QoE investigations (adaptive point cloud streaming). We will also put the work into the context of international multimedia systems research.
What Not to Document and Why_ (North Bay Python 2024)Margaret Fero
We’re hopefully all on board with writing documentation for our projects. However, especially with the rise of supply-chain attacks, there are some aspects of our projects that we really shouldn’t document, and should instead remediate as vulnerabilities. If we do document these aspects of a project, it may help someone compromise the project itself or our users. In this talk, you will learn why some aspects of documentation may help attackers more than users, how to recognize those aspects in your own projects, and what to do when you encounter such an issue.
These are slides as presented at North Bay Python 2024, with one minor modification to add the URL of a tweet screenshotted in the presentation.
What's Next Web Development Trends to Watch.pdfSeasiaInfotech2
Explore the latest advancements and upcoming innovations in web development with our guide to the trends shaping the future of digital experiences. Read our article today for more information.
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Chris Swan
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
Transcript: Details of description part II: Describing images in practice - T...BookNet Canada
This presentation explores the practical application of image description techniques. Familiar guidelines will be demonstrated in practice, and descriptions will be developed “live”! If you have learned a lot about the theory of image description techniques but want to feel more confident putting them into practice, this is the presentation for you. There will be useful, actionable information for everyone, whether you are working with authors, colleagues, alone, or leveraging AI as a collaborator.
Link to presentation recording and slides: https://bnctechforum.ca/sessions/details-of-description-part-ii-describing-images-in-practice/
Presented by BookNet Canada on June 25, 2024, with support from the Department of Canadian Heritage.
Sustainability requires ingenuity and stewardship. Did you know Pigging Solutions pigging systems help you achieve your sustainable manufacturing goals AND provide rapid return on investment.
How? Our systems recover over 99% of product in transfer piping. Recovering trapped product from transfer lines that would otherwise become flush-waste, means you can increase batch yields and eliminate flush waste. From raw materials to finished product, if you can pump it, we can pig it.
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsMydbops
This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization.
Key Takeaways:
* Understand why connection pooling is essential for high-traffic applications
* Explore various connection poolers available for PostgreSQL, including pgbouncer
* Learn the configuration options and functionalities of pgbouncer
* Discover best practices for monitoring and troubleshooting connection pooling setups
* Gain insights into real-world use cases and considerations for production environments
This presentation is ideal for:
* Database administrators (DBAs)
* Developers working with PostgreSQL
* DevOps engineers
* Anyone interested in optimizing PostgreSQL performance
Contact info@mydbops.com for PostgreSQL Managed, Consulting and Remote DBA Services
Performance Budgets for the Real World by Tammy EvertsScyllaDB
Performance budgets have been around for more than ten years. Over those years, we’ve learned a lot about what works, what doesn’t, and what we need to improve. In this session, Tammy revisits old assumptions about performance budgets and offers some new best practices. Topics include:
• Understanding performance budgets vs. performance goals
• Aligning budgets with user experience
• Pros and cons of Core Web Vitals
• How to stay on top of your budgets to fight regressions
In this follow-up session on knowledge and prompt engineering, we will explore structured prompting, chain of thought prompting, iterative prompting, prompt optimization, emotional language prompts, and the inclusion of user signals and industry-specific data to enhance LLM performance.
Join EIS Founder & CEO Seth Earley and special guest Nick Usborne, Copywriter, Trainer, and Speaker, as they delve into these methodologies to improve AI-driven knowledge processes for employees and customers alike.
5. @paigerduty
1
2
3
4
5
6
7
8
9
10
11
12
13
14
The Zen of Python (aka PEP 20)
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!