Show HN: LLM Based Spark Profiler https://ift.tt/j378hIO

Show HN: LLM Based Spark Profiler Hey HN, Spark event logs run into 100s of MBs and offer a wealth of insight into your workloads but making sense of them has always been quite a bit prohibitive. We’ve recently built a lightweight tool that automatically parses Spark event logs and surfaces targeted insights to help you optimize your data jobs. Whether you’re chasing down a bottleneck or balancing performance vs. cost, the profiler got you covered with real-time configuration recommendations, data skew analysis, and more. Curious how it works in action? Check out this quick Loom video for a walk-through: https://ift.tt/Sgi6dEu?... We’d love your feedback — check it out at https://app.datasre.ai and let us know what you think! https://datasre.ai/ April 9, 2025 at 09:42PM