Subscribe
Sign in
Qodo (formerly CodiumAI)
p/codiumaiAgentic AI for testing, reviewing, and writing code.
Start new thread
Ben Lang

CodiumAI - Generating meaningful tests for busy devs

by
Top Product
•

CodiumAI analyzes your Python, JS, or TS code and generates meaningful tests to catch bugs before you ship. With CodiumAI, you can easily and quickly generate comprehensive test suites that help you ensure the reliability and correctness of your software.

Add a comment

Replies

Best
Shai Geva
Maker
📌
Hey guys, I'm with the CodiumAI team, I made a few videos to introduce some of our features. Just a taste of what CodiumAI can do :) Basic showcase: https://www.loom.com/share/658d0... Basic configuration options: https://www.loom.com/share/22f2d... Reference tests: https://www.loom.com/share/a607c...
Itamar Friedman
Maker
@shai_g thanks! Great share! One of our users also created a video showing with an excellent example of using CodiumAI to generate tests for the Gilded Rose kata (typescript):
Itamar Friedman
Maker
📌
Hello Product Hunt ! 🚀 I'm thrilled to announce the launch of CodiumAI [beta], an innovative solution to generate component testing of your Python, JS, or TS code. 🧪 Testing your code is important. Aside from catching bugs, it gives you valuable insight into your code, and lets you know you’re coding with a purpose. 🌰 But writing non-trivial test cases is tedious, frustrating, and time consuming. And when bugs still pop up, it can be hard to figure out how you missed that edge case. In retrospect, it’s so obvious! 🤝 As devs, we get it. And that’s why we made CodiumAI. As a veteran developer and R&D manager, I'm excited to present this powerful tool that can generate comprehensive test suites for your software quickly and easily, within your IDE. ✅ CodiumAI generates meaningful tests. Tests that actually check the functionality of your code – and suggests them right inside your IDE, as you type. It does this by analyzing and deeply understanding your code, docstrings, and comments, and by interacting with you, the dev. So if you want code that functions like you meant it to, get CodiumAI for meaningful test generation. Try CodiumAI today! You can find CodiumAI / TestGPT in VS Code or JetBrain marketplace. 🙏 We greatly value your input and take all feedback into consideration as we aim to deliver the best possible solution for your software testing needs. Please drop your comments and questions below, we'll be here all day and are excited to hear from you! Itamar, the CEO and Co-founder of CodiumAI
Dean Šoć
@maritamar Is Java somewhere in the pipeline?
Itamar Friedman
Maker
@dean_soc , definitely. Probably next in line. Although, recently, on our discord server we got more requests for Ruby..
Ben Lang
Top Product
Hunter
📌
Congrats Itamar and team on the launch!
Itamar Friedman
Maker
@benln , thank you Ben! We are lucky to have such an excellent Hunter! We are just Gathering feedback here :)
Eddie Forson
Congrats on the launch! This looks super useful as we have a growing Python codebase and I'm not always doing the best job when it comes to writing tests due to lack of time... Have installed Codium and will give it a go 👌🏿 What are the next steps for this project?
Tal Ridnik
Maker
@ed_forson Hi Eddie. In a week or two, we plan to upgrade the Pycharm GUI, and add capabilities that are currently only in our VS-Code extension. For example, auto-mocks, example input, example test, give-me-more,... So if you are a Python-PyCharm user, stay tuned :-) Under the hood, a lot is going on. We will definitely continue to improve the quality of our tests, enable them to have a larger context, and add capabilities and features.
Almog Lavi
Maker
@ed_forson Thank you, Eddie! I hope our product will benefit you. Next, we are planning to support more languages, and more IDEs, we will, of course, keep improving the test results (getting feedback from users will certainly help!) and we are planning to add Team's plan with more advanced capabilities. I invite you to register for our newsletter and stay tuned https://www.codium.ai/#home-wait 🌻
Itamar Friedman
Maker
@ed_forson , thank you for asking! 1> We wish to make sure that almost any type of code (different frameworks, different languages) is well supported by CodiumAI. 2> We want to improve our UX/UI according to feedback we hear from our dear users. These are the top urgent items in the roadmap. In our seed funding announcement, we share a longer roadmap of our plans! https://www.codium.ai/blog/codiu...
Nevo Alva
This is really awesome. I'll let our team try it and let you know how it went. Can you share some results so far? like do devs trust this?
hussam lawen
Maker
@nevoalva Hi Nevo, Thank you for your question. We constantly benchmark on HumanEval Test Generation Task among other benchmarks internal benchmarks. On HumanEval Test Generation, CodiumAI's TestGPT delivers tests with higher Branch Coverage compared to simple Test completion with GPT3.5-Turbo (97.3% vs 92%) and a higher test pass rate of 97% vs 61%.
Razii Abraham
That's interesting! I'm not a tech person, but as Product Manager I know that `testing` is one the most crucial steps in shipping a product/feature. Glad if there's a tool to help the engineers (and testers) to prevent a `hotfix/rollback` required once a bug/defect found in the production env. Good luck!
Itamar Friedman
Maker
@razii_abraham , thank you! By the way, in the long term, we hope to introduce the concept of "executable specification". In such a future, Product Managers will also be able to test code behavior. You can read a bit about it here: https://www.codium.ai/blog/codiu...
Farooq (SF Ali) Zafar
Congratulations @maritamar on your launch! Brilliant idea to leverage TestGPT to generate test cases, catch bugs, and check code. Looking forward to putting this through the paces!
Itamar Friedman
Maker
@sfali789 , thank you! We really want to help developer to make code logic testing a less tedious and frustrating task... it could even be a fun experience.
philip tannor
Haha I was wondering when we’d see the first app of this genre. Can’t wait to have my team try it!
Itamar Friedman
Maker
@ptannor , thank you Philip. I really appreciate such an endorsement from the maker of https://deepchecks.com/ ! We would love to hear your team feedback!
Tal Ridnik
Maker
Thanks @ptannor Will be glad to hear your team's feedback about the tool :-)
Anil Matcha
Writing tests is a painful problem for us devs. Generating it via AI will be very useful
Itamar Friedman
Maker
@matcha_anil , definitely ! It could even be fun. I just tried giving CodiumAI this general instruction: "Add funny, pirate-like, docstrings. Make them different for each test." and got this test: >>> # Tests that a call is dispatched to an available operator. tags: [happy path] def test_dispatch_call_to_operator(self): """ Ahoy matey! Let's test if a call can be dispatched to an available operator. If there be an operator available, the call should be taken and the test should pass with flying colors! """ operator1 = Operator(1, "Jack Sparrow") operator2 = Operator(2, "Blackbeard") call = Call(Rank.OPERATOR) call_center = CallCenter([operator1, operator2], [], []) call_center.dispatch_call(call) assert call.employee == operator1 or call.employee == operator2
Daniel Shir
Do you guys use Codium to test Codium? ;P
Itamar Friedman
Maker
@daniel_shir_acumen , haha, of course! That is one of advantage of building a dev tool, and it is fun! It doesn't mean we do not have bugs! Code Integrity is about developers’ confidence in the correctness of their code. We wrote some thoughts here: https://www.codium.ai/blog/codiu...
Mahmudul Hasan
Congratulations 🎉🎉🎉
Itamar Friedman
Maker
@hasan_ui_design , thank you! We are very excited -- as developers ourselves, we felt the pain with code logic testing, and wanted to deal with it once and for all
Tajul Islam
Congratulations on the launch of CodiumAI! It sounds like a very promising tool that can save developers a lot of time and frustration when it comes to testing their code.
Itamar Friedman
Maker
@tajul_islam1 , thank you! Exactly, we want to help getting rid of this frustration. As developers, we get it -- we also suffered from it for years!
Ron Fybish
Love it 😍
Itamar Friedman
Maker
@ron_fybish1 , thank you! Happy to hear! We would love to hear any feedback or suggestion!
Karan bhansali
Generating test cases using AI sounds interesting, writing test cases is a challenge. Congratulations team codium.
Itamar Friedman
Maker
@karan_bhansali , agree! Generating meaningful tests is a challenging task, even with the latest AI capabilities! And we are serious about it. We're so excited to announce our first milestone today, and we'll keep adding capabilities and features, and improving our product accordingly.
Isha Varma
It is tough to write tests for your own code. I am too lazy to do it most of the time. Love the product. Congratulations on the launch!
Itamar Friedman
Maker
@isha_varma07 , thank you Isha. We hope to make it much faster and more enjoyable for you to generate a meaningful test suite. Let us know how it worked!
Kushagra Sharma
Great product and congratulations on the launch. Some questions: 1. Does it have built in support for in memory DBs to mimic the actual calls ? 2. Are there any plans to support writing integration tests by chaining calls together ? As a developer, I would love to see something which only needs the chaining instructions (in a yaml, json) and have it produce end to end tests
Shai Geva
@kushagra_sharma3 Hi Kushagra, Thank you! Very good questions. 1. The way CodiumAI can currently help with this is by learning from your existing tests. If you already have tests that work this way, you can provide them as reference tests and Codium will pick up the initialization and style. Please see this video for an example (with JavaScript. Python behaves the same way): https://www.loom.com/share/a607c.... Does that help in your use case? 2. At this point in time we don't have specific plans for such a feature, but I would be very happy to hear more about the use-case! What type of integration tests were thinking about? Tests that run through a standard test runner like pytest / jest, and make HTTP request to a configured backend? Thanks, Shai
Kushagra Sharma
@shai_codium 1. Alright, makes sense 2. Historically I have seen integration tests where calls are either chained between modules/files(like Unit tests) or chained as http calls to the backend sitting outside the service(s). We have seen that CodiumAI can create tests for something like create an order, cancel of an order, refund an order. An integration test would typically be something like Create an order -> Pick inventory(validate) -> Generate shipping label(validate) -> Ship an order(validate). The orchestration of calls is the key here and there can be a number of such call flows. Looking for a low code/no code solution for this
Shai Geva
@kushagra_sharma3 Hi, sorry for the delay, Thank you for the info! I think in this case CodiumAI might be able to help - but we are currently focused on test-code generation, so not as a low code/no code solution, only as standard code-based tests. Thank you again, Shai
Noa Green
Sounds super interesting!
Itamar Friedman
Maker
@greenoa , thank you Noa! Don't hesitate to send over any suggestion or idea! We are serious about code level testing and Code Integrity in general. Any thought on this topic will be considered!
Ran Ribenzaft
Congrats team! Looks super promising 🎉🚀
Itamar Friedman
Maker
@ranrib , thanks Ran! We hope that already with our beta version, developers can generate meaningful tests right inside their IDE with much less frustration than before. Some alpha release users even said that some generated tests felt magical for them. But we defiantly have a long way to go, we are just starting!
Yakov Mamontov
What other languages are you planning to add in the near future?
hussam lawen
Maker
@capator Java/Kotlin, However, we'd love to hear from you about what other languages you'd like us to add next.
Jim Engine
Congratulations @maritamar on launching Codium! 🎉 It seems like an amazing tool that can help busy developers generate meaningful tests effortlessly. I love the idea of getting non-trivial tests suggested right inside the IDE, making it easier for developers to code smart, create more value and stay confident when pushing their code. The fact that Codium analyzes code, docstring, and comments, and interacts with you to suggest tests is really impressive. It's great to see that it provides full visibility of how the code behaves and how the changes affect the rest of the code. And by the way, I like the landing page.💪
Itamar Friedman
Maker
@officialexaking , thank you Jim! We hope to meet your expectations, let us know about anything you have in mind: suggestions, issues, requests..