John Janiczek

2 things: - become FAANG engineer - make 3,251xxx+ per year Both goals are worth admiration but not tied to each other. Your income not always tied to your qualifications and you may be prone to survivor bias. We may never know the stories of those other amazing engineers with same traits that did not get there.

14

1 Comment

NVIDIA has just launched the NVIDIA TensorRT Model Optimizer enabling Inference optimization techniques like quantization and sparsity now publicly available! This comprehensive library of post-training and training-in-the-loop optimization techniques can: - Reduce model complexity with quantization and sparsity techniques - Optimize inference speed for deep learning models - Integrate seamlessly with PyTorch, ONNX, TensorRT-LLM, and more ✨ Highlights from the blog: - Advanced calibration algorithms like INT8 SmoothQuant and INT4 AWQ - Record-breaking inference speedups for LLMs and diffusion models - Cutting-edge quantization techniques, including INT4 QAT for the next-gen NVIDIA Blackwell platform - Composable APIs for stacking multiple optimization steps effortlessly 🛠️ Now available for installation as a PyPI wheel, the Model Optimizer empowers you to achieve unprecedented performance. Check out the blog for more details and start today. Let me know if you have any requests our team can help. #GenerativeAI #DeepLearning #InferenceOptimization #NVIDIATensorRT #ModelOptimizer

79

1 Comment

🚀 Announcing #OCTUNE: Autonomous (Aerial) System Auto-Tuning – Now #OpenSource! 📊 We are excited to unveil #OCTUNE. This cutting-edge #autotune algorithm, now open-source, revolutionizes #PID controller tuning for autonomous systems, notably #UAVs operating on #PX4 autopilots. Contributors: Dr. Mohamed AbdelKader, Mohamed Mabrok, Anis Koubaa 🔧 What is OCTUNE? OCTUNE (Online Optimal Control Tuning) allows for real-time, automatic tuning of PID controllers with no need for prior knowledge of the plant model. This innovation leads to enhanced performance and adaptability under varying conditions and configurations. 🎥 See OCTUNE in Action: Real quadrotor tuning: Watch on #YouTube [https://lnkd.in/gFrQ7Jss] Simulation in ROS Noetic & Gazebo: Watch on YouTube [https://lnkd.in/gXQV2kd2] 🌐 #OpenSource Release: OCTUNE is ready for integration into your projects. It’s perfect for anyone working with drones, robots, or autonomous systems seeking to improve system responsiveness and stability. 🔬 #Research & #Validation: Our findings have been thoroughly validated through simulations in Gazebo and real-world experiments on quadrotor platforms. Fast, stable, and reliable tuning is now achievable in minutes! Read our paper here: https://lnkd.in/gyJwjYYs 🛠️ Get Started: Explore OCTUNE on GitHub: OCTUNE Algorithm [https://lnkd.in/gFgNu8EY] For ROS and PX4 users: Repository Link [https://lnkd.in/gFgNu8EY] NVIDIA Jetson Setup: OCTUNE Jetson Setup [https://lnkd.in/g6RkPjsv] 💡 We Need Your Input! Help us refine and expand OCTUNE by contributing to our community-driven development. Your insights are invaluable in pushing the boundaries of what autonomous systems can achieve. 🤖 Join our community of innovators and leverage OCTUNE in your projects to make autonomous control systems more adaptive and efficient. Your expertise could spark the next big breakthrough! #Robotics #OpenSource #AutonomousSystems #PIDController #UAVTechnology #Innovation #OCTUNE #PX4 #ROS #Gazebo #NvidiaJetson

42

1 Comment

Yet another innovative paper on State Space Models https://lnkd.in/eevz_GiE achieves arbitrarily low memory decay (meaning arbitrarily long memory) by filtering the input sequence with a precomputed (not learned) spectral filter. Recurrent models, such as SSMs, have a general form of x_t = A x_t-1 = A^2 x_t-2 = … = A^t x_0. Because of the power term A^t, they model nonlinearity well. However, this comes with a cost, ||A|| has to be < 1. Otherwise, the model is “explosive” and will grow out of control rather quickly. For any A satisfying ||A|| < 1, A^t goes to 0 very quickly, meaning the effect of x_0 on x_t diminishes rapidly, which is known as “memory decay”. This limitation is why SSMs may not outperform transformers in the needle-in-the-haystack test. That is until this paper. And the trick? View the signal in its waveform. If something occurs repeatedly without decay, it must be a wave. By using a spectral filter that transforms signals from the time domain to the frequency domain, the paper proves that Spectral SSMs can have a memory longer than any given L. And the best part? The spectral filter is precomputed, not trained. Like any other SSMs, a hybrid architecture leads to better performance. Intuitively, this is because some signals can be efficiently expressed in the frequency domain, while some other signals can be efficiently expressed in the time domain, and having both helps. #artificialintelligence #machinelearning #deeplearning

74

3 Comments

Take a look at newest improvements of our #OpenVINO: 1. Better LLM service using OpenAI-compatible APIs 2. Enhanced GenAI performance on both CPU and GPU devices 3. Support for INT4 GPTQ 4. Our new C++ Generate API for more efficient text generation 5. Ongoing batching for LLMs

12

PyRoboSim 2.0.0 is out! Major changes include: * Ubuntu 24.04 / ROS 2 Jazzy support * Replaced PyQt5 with PySide6 for better licensing and cross-platform support * Use ROS 2 actions instead of topics for commanding actions + task plans * New Trajectory class to replace loose arrays * Lots of improvements to CI and Docker images (thanks Erik Holum!) Full changelog: https://lnkd.in/euZ-CBiu

226

2 Comments

Julia SerDes part 4 showcases a more detailed implementation example of the transmitter model. We finally get to do some real SerDes modeling! https://lnkd.in/gPspMStB I can't recommend Pluto notebook's reactivity enough. It makes playing around with SerDes parameters much more responsive🚀 and fun🎉. Run the notebook, change a transmitter parameter and see the eye diagrams change in real time! (almost like looking at oscilloscopes 👀) Edit: Forgot to mention that with some bug-fix and optimizations, the example model now finishes 1-million bits in around *7 seconds*. If only FFTs can be even faster. #Julia #SerDes #Simulation #System #Modeling #CircuitDesign #OOP #opensource

32

How the hidden internal state of CLIP changes through increasing pixel occlusion? I am working on understanding how different preprocessing (for the same input image) affects the internal representation for some computer vision models. This is motivated by subtle differences between torchvision and Nvidia DALI (which uses nvJPEG). But before reaching there, I am testing how simple occlusion affects the embedding (some idea of sensitivity). In this colab (https://lnkd.in/e5x2BTWT) we see how the cosine similarity of CLIP embeddings decrease linearly with the number of pixels replaced by a grey pixel. Next experiment will test if the difference of the cosine similarity comes from a certain set of features that are consistently changing its value (i.e. irrelevant features) #CLIP #ComputerVision #embeddings

2

LLMs and foundation models are currently HUGE, so they're nearly impossible to run on embedded devices. But did you know there are ways to obtain all the benefits of LLMs even when deploying to microcontrollers? 🤯 In this recent talk, I explain how to transfer the core capabilities of LLMs onto even the tiniest of models: - Zero-shot learning - Reasoning - Information retrieval - Data generation We lean heavily on this approach in our work at Edge Impulse. Watch the session to learn more! https://lnkd.in/eDYzWJku

67

1 Comment

In-Context Learning at Extreme Scales: My Thoughts As large language models (LLMs) continue to expand their context capacities, the domain of in-context learning (ICL) at extreme scales has piqued my interest. A recent paper by researchers at Carnegie Mellon University and Tel Aviv University provides a detailed exploration into this area, unveiling both impressive capabilities and notable challenges. 🔍 Key Insights: Performance Improvement: The study demonstrates that ICL can improve significantly by integrating hundreds to thousands of examples into the context window—far beyond traditional few-shot settings. This is particularly impactful in complex tasks like 77-way banking intent classification, where long-context ICL matches or surpasses traditional finetuning. Input Sensitivity: Long-context ICL shows reduced sensitivity to input shuffling compared to shorter contexts. However, organizing examples by label dramatically reduces performance, emphasizing the need for diverse context configurations. Source of Gains: Performance gains primarily arise from accessing more relevant examples, rather than aggregating knowledge across many examples during encoding. 📉 Challenges & Drawbacks: Sensitivity to Example Order: Despite reduced sensitivity, the performance still varies with input order, indicating potential instability in practical scenarios. Performance Saturation: The models often do not fully leverage their extended context capacities, showing performance saturation before reaching the maximum context length. Computational Cost: The increased computational cost associated with using large context windows could limit practical deployment, particularly in resource-constrained environments. Efficiency Issues: There is a noted inefficiency in how models utilize the encoded information within large datasets, suggesting that current architectures may need refinement to better use available data. 🚀 Future Directions: The paper suggests several exciting avenues for future research, including optimizing the trade-offs between computational costs and performance, exploring model adaptation strategies, and refining ICL approaches for specific applications or datasets. As we push the boundaries of what LLMs can achieve, effectively leveraging their full context could unlock new capabilities. This area is ripe for further exploration and could lead to major advancements in how we deploy AI. For those at the cutting edge of language model development, I highly recommend this paper for its thorough analysis and forward-looking perspective. Check out the full work at https://lnkd.in/d7AmGh4y. What are your thoughts on the future of in-context learning at such scales and the challenges discussed? #AI #MachineLearning #DeepLearning #LanguageModels #InContextLearning

17

I'd like to highlight the paper "N-dimensional Gaussians for fitting high-dimensional functions" by Stavros Diolatzis et al. that will be presented at SIGGRAPH 2024. It offers enhanced flexibility and robustness compared to traditional neural networks approaches while maintaining fast training times, accommodating additional input dimensions, addressing the limitations of explicit methods in representing complex visual appearances and view dependent effects. The method demonstrates significant improvements in training efficiency and inference performance through innovative techniques like refinement process controlled by the optimizer and efficient Gaussian culling. For more details, the blog post and the paper are available at https://lnkd.in/gb-wnyUe. #Intel #Graphics #Research #Gaussians

8

New ODE (ordinary differential equation) solvers in #MATLAB from the SUNDIALS suite, provides the ability to perform parameter sensitivity analysis. Learn more here 🔗 https://spr.ly/6045gZu75

2

Paco, thank you so much for joining Ron and me for this fascinating discussion about graphs in artificial intelligence! So much fun to join in this KUNGFU.AI Hidden Layers episode!

11

1 Comment

Although I appreciate the various approaches to Theory of Mind (ToM), it is worth noting that there is still no unified mathematical theory that consolidates all the theories regarding decision-making in the brain. In my opinion, there is indeed consciousness in decision-making, especially due to the functions of the hippocampus, which processes at a high level what has been decided at the stimulus-response level, akin to what the hypothalamus does. The hippocampus plays a fundamental role in learning and orchestrates a large part of it. Additionally, decisions influence each other, which would require a certain level of consciousness. Furthermore, according to ToM, there is a social and feedback component inherent in decision-making.Lastly, memory is subjective and enriched by prior experience and lessons learned, which also necessitates consciousness.

8

Yann LeCun to Elon Musk: "Curious to know how you could possibly do real-time camera image understanding in FSD without ConvNets, TBH." An interesting question, is it not? I am old enough to remember when edge detection and feature extraction and segmentation was done via traditional engineering - by carefully examining the domain and developing domain-specific and computationally efficient models and then testing and trying and tweaking and tinkering until it all worked. And once the relevant features are extracted, further "image understanding" does not actually need ConvNets, no? Other ideas?

3

5 Comments