AI In Code Reviewing

We all know that a little context can completely change the answers to our AI models. And this is not only limited to AI even humans operate in a similar way. The same principle applies to coding. Without context, code reviews can miss the mark. Traditional approaches often focus on surface issues while deeper, more complex problems go unnoticed.

That’s where AI-based code reviews come in. By adding advanced artificial intelligence, these reviews can bring a deeper understanding of the issues, changing a routine check-up into a strategic asset. AI-based code reviewers can completely change how we develop our future projects. So, without further ado, let’s dive deep into AI code Reviewers.

What Are AI-Based Code Reviews?

No one can write the perfect code in the first go; especially when it comes to enterprise solutions. Code reviews are a must to ensure that the production code runs smoothly and efficiently.

Traditionally code reviews involve developers manually inspecting each line of code to catch errors, enforce standards, and suggest improvements. While effective, this process is often time-consuming, prone to human error, and inconsistent. Having multiple reviews adds even more complexity.

AI-based code reviews elevate this process by leveraging machine learning and advanced algorithms to automate and enhance the review process. These intelligent systems not only identify syntax errors but also understand the context in which the code operates, offering insights that go beyond surface-level issues.

The Depth of AI-Based Code Reviews

AI-driven tools analyze the project’s goals, the functionality of other code components, and how recent changes might impact the overall system. For instance, when reviewing APIs, AI can assess compatibility with existing systems, evaluate infrastructure impact, and ensure adherence to best practices. This type of view ensures that every line of code integrates properly with the entire system, aligning perfectly with the project’s objectives.

Common factors considered in AI-based code reviews include:

• Adherence to Coding Standards: Ensuring code follows project-specific conventions, such as naming conventions and directory structures.
• Impact on Existing Code: Assessing whether changes introduce bugs or require updates to documentation.
• Infrastructure and Performance: Evaluating potential impacts on system performance and infrastructure requirements.
• Security and Robustness: Identifying potential vulnerabilities and ensuring new additions are secure.
• Consistency and Optimization: Maintaining consistency with existing APIs and optimizing solutions for the problem at hand.

Since AI can look at the entire code base in one go, these reviews can truly improve the practices of software development, making sure that each line of code performs well and integrates flawlessly with the entire system.

But don’t just rely on the AI, after all, it is not perfect, and it can surely make mistakes. But for most cases, these automated AI reviews can come in really handy. AI struggles a lot especially when you are doing something novel and that’s where you should always double-check things.

The Difference from Traditional Code Reviews

Traditional code reviews are somewhat like proofreading a document for grammar and spelling errors without worrying about the story or the intent behind it. Useful, yes, but potentially missing bigger issues like plot holes or character inconsistencies.

Code reviews typically follows a structured approach: one developer authored the code and passed it to another for evaluation. The reviewer thoroughly examined the submission, analyzing each line and providing notes, comments, questions, or identifying potential issues. The process generally unfolded as follows:

1. Pull Request Creation: A developer creates a Pull Request (PR), documenting the changes made and requesting a review.

2. Review Process:Another developer or an engineering manager reviews the PR, focusing on specific criteria:

• Ensuring the changes align with the scope of the associated ticket(s).
• Verifying the correctness of business logic implementation.
• Upholding or improving code quality by reverting unnecessary changes and adhering to coding standards.
• Checking for potential security vulnerabilities.

3. Approval or Rejection:

• Approval: If the code meets all standards, the reviewer approves the PR for merging.
• Rejection: If issues are found, the reviewer provides detailed feedback on the necessary updates.

4. Iteration:The developer revises the code based on the feedback, and the process repeats until approval is granted.

This workflow has become a cornerstone of modern development, ensuring collaboration and quality. It hinges on the reviewer’s availability to offer feedback and the developer’s responsiveness to implement changes.

AI-based code reviews, in contrast, view the code as part of a larger picture. They not only identify straightforward errors but also uncover subtler issues that could affect the application’s behavior in specific scenarios. This approach ensures higher code quality, faster development cycles, and enhanced security.

Challenges of Traditional Code Reviews

1. Time-Consuming: Manual reviews can become bottlenecks, delaying the shipping of code.
2. Human Error: Reviewers may overlook subtle bugs or security risks, especially under tight deadlines.
3. Inconsistency: Different reviewers might have varying standards and focus areas, leading to inconsistent feedback.

AI-based code reviews address these challenges by providing consistent, unbiased, and rapid assessments, freeing developers to focus on more complex and strategic tasks.

Powered by AI and Machine Learning

AI and machine learning are the driving forces behind the intelligence of AI-based code reviews. These technologies allow review tools to learn from past code, recognize normal patterns, and predict potential problems based on vast amounts of data. Their primary focus isn’t just about catching a forgotten semicolon; it’s about understanding how a small change could ripple through your system in unexpected ways.

Most likely these code review assistants are based on some type of LLMs, and given that these LLMs have seen so much data, it is quite easy for them to understand what a good looks like and what are the most common errors.

AI Tools in the Code Review Landscape

There are many AI-powered tools and platforms offering this service. A few notable among them are:

• GitHub Copilot: Assists in code generation and provides real-time suggestions.
• CodeGuru by Amazon: Focuses on identifying performance issues and recommending improvements.
• DeepCode by Snyk: Uses AI to analyze code for potential bugs and vulnerabilities.

These tools leverage machine learning and advanced algorithms to automate processes, suggest optimizations, and even generate code snippets to address identified issues. Adoption of such tools is definitely going to be more and more with each passing month.

CodeRabbit

While the landscape of AI-based code review tools is growing, CodeRabbit might be quite a strong competitor to Copilot and tools from other big labs.

CodeRabbit integrates AI into the review process, serving as both a guide and an assistant. Here’s how CodeRabbit elevates context-aware code reviews, providing features that not only streamline but also significantly enhance the development process:

Key Features of CodeRabbit

1. Pull Request Summaries: Automatically generates comprehensive summaries of pull requests, breaking down changes by file or directory. This allows developers to quickly grasp modifications without the tedious process of manual testing and iteration.
2. Chat with Code: Enhances interactivity by allowing developers to engage directly with the tool through a chat interface. Developers can ask for detailed explanations about suggested changes, propose alternatives, or provide corrections to enhance the tool’s learning.
3. In-Depth Code Reviews: Conducts thorough, incremental reviews with each new commit, meticulously examining every piece of code. It spots potential issues, bugs, or vulnerabilities that might be missed otherwise, offering detailed explanations and actionable suggestions.
4. Committable Suggestions: Includes a feature that allows developers to apply suggested changes directly, minimizing the risk of errors advancing to production and ensuring a higher quality final product.

CodeRabbit is not just a simple wrapper that does pass through to the LLM models. To circumvent context size limits, CodeRabbit uses an innovative, multi-LLM, and multi-stage approach to scale reviews for larger change sets. Unlike AI-based code completion tools, code reviews are a much more complex problem. The reviewer context is much broader than the developer context, as the reviewer needs to uncover not just obvious issues but also understand the larger context of the pull request and changes across multiple files.

Since LLMs are limited by their context window. CodeRabbit uses a clever technique to handle this and provide comprehensive answers.

Summarization Layers:

• Generate summaries for each file and prioritize critical context.
• Recursive summaries distill relevant details, ensuring larger change sets fit into the LLM’s context window.

Incremental Reviews: Analyze changes incrementally (e.g., by files, hunks, or commits) to stay within token limits while retaining meaningful context.

CodeRabbit vs. Competitors

While tools like GitHub Copilot, CodeGuru, and DeepCode offer valuable functionalities, CodeRabbit differentiates itself through its comprehensive feature set and seamless integration capabilities:

• Comprehensive Summaries: Unlike some tools that focus primarily on code generation or performance issues, CodeRabbit provides detailed pull request summaries, saving developers significant time.
• Interactive Interface: The chat-based interaction sets CodeRabbit apart, making the review process more engaging and insightful.
• Advanced Integration: CodeRabbit seamlessly integrates with issue management tools like Jira and Linear, enhancing issue tracking and project management.
• Continuous Learning: CodeRabbit continuously learns from interactions, improving its feedback and suggestions over time, ensuring that it remains aligned with your project’s evolving needs.

I personally think integration with JIRA, and summaries of PRs might be a game changer for the entire industry.

Advantages of AI-Based Code Reviews

These advanced reviews do more than just enhance code quality — they change how code is developed and significantly influence project outcomes:

• Spot Subtle Discrepancies: By looking at the complete project in one go, AI-based reviews can detect subtle discrepancies that might otherwise go unnoticed, such as interactions between modules that could lead to unexpected bugs.
• Accelerate Development Cycles: With their efficiency and precision, AI-based reviews streamline the development process, reducing the cycle of feedback and revisions, and allowing developers to progress faster.
• Improve Software Health and Longevity: Ensuring that new code integrates seamlessly with existing systems helps maintain a clean and scalable codebase, enhancing software reliability and ease of future enhancements.
• Enhance Security Proactively: AI-based reviews address potential security vulnerabilities by considering how changes affect the overall system, preventing security issues before they become threats.
• Tailor Developer Support: Beyond identifying issues, AI-based reviews provide personalized feedback, fostering a learning environment that encourages continuous improvement and skill enhancement.

Two of the most important tools modern developers rely on to improve their code are static code analyzers (SCAs) and AI code reviewers.

SonarQube is probably the most popular SCA in the software industry. Its primary strength lies in its rule-based analysis, which efficiently spots syntactic errors and standard violations.

ButAI reviewers like CodeRabbit go one step further and not only show the syntax errors but also use their contextualization to solve other errors and recommend things for better efficiency.

CodeRabbit In Action

At a brief glance, you can see the PR contains two code commits with changes across six files. Instead of clicking through the six files and reviewing code changes, there’s a quick summary of the new features and enhancements being introduced.

File-level changes are summarized in a short table followed by another table that demonstrates whether the code changes actually address existing software requirements. The maintainer doesn’t need to manually review whether the feature requirements are actually addressed. Everything is spelled out in a way that can be visually skimmed for completeness.

In particular, we want to highlight the “codebase verification” feature that happens near the end of the PR.

The AI detects a reference to an old method (GetHandler) and finds that “not all references to the method were updated following its renaming to bGetHandler in the Router struct.” Perhaps this updated function name was a typo that needed correction, or perhaps it was an intentional renaming that wasn’t consistently applied. In either case, this could have been a breaking change introduced into the codebase that was caught by the AI.

In some cases, it might generate things that you might not want and could be a little unnecessary. For instance, the Poem feature is something that a lot of people might not like because it just adds to the noise.

But all in all, I think that AI code reviewers could be a great benefit to your software development process. But use it carefully, it could make you lazy and might lead you to believe AI too much. We are still far from here.

There would be many cases where it will fail, especially when it comes to novel use cases. Because that’s where LLMs fail and in the end, all AI code reviewers are using LLMs only to suggest changes. So, if LLMs fail on those problems so will the AI code reviewers.

Conclusion

AI-based code reviews represent a significant leap forward in software development, offering enhanced efficiency, accuracy, and consistency. By integrating AI into the code review process, teams can achieve higher code quality, faster development cycles, and improved security, all while fostering a collaborative and continuous learning environment.

My only advice is to use any AI tool wisely, if not configured or used properly they can lead to even more errors. Do not become lazy and do check AI-generated code for correct functionality. AI is only as good as your explanation of the problem.