Sitemap
Open in app

Sign in

Medium Logo
Write

Sign in

Automating MR review on Self-hosted Gitlab using AI(Ollama)

Lee young-jun
9 min readApr 23, 2025

Catch two rabbits easily and securely

One day, my manager asked if there was a way to improve workflow using AI.

I had an idea: automate Merger Request(MR) review with AI.

Since our repositories are hosted on self-hosted Gitlab instance, I needed to set up a Gitlab-runner to execute any CI pipelines.

However I didn’t have permission to access repository settings.

A few days later, the manager suddenly brought up the idea and assured me that he would provide everything I needed.

So I began the task. Let’s start!!!

Gitlab-runner

As I mentioned above, we need a Runner to run CI jobs on Gitlab.

Installing gitlab-runner

It is very easy to install runner. Just run this command:

brew install gitlab-runner

Ah.. sorry I set up this environment only on MacOS.
You should ask GPT how to get Runner if you are on another OS.

Registering the Runner

The Runner receives CI jobs and process them. However it dosen’t know which repository to access, and we should also grant the permission.

To register the runner into your Gitlab repository, go to settings and get token for the runner..

gitlab-runner register --url {repository url} --token {gitlab token}

Enter root repository url, not the project’s.

Running runner on background

Now we can run the runner.

gitlab-runner run

However it stops after rebooting your machine.
So instead, run with start argument

gitlab-runner start

If you can’t do this due to any error.
Try this instead.

brew services restart gitlab-runner

This will register gitlab-runner service on your machine.

Gitlab CI

CI

CI is Continuous Integration. It means the script to check if new changes are Ok to merge. For exmaple) testing, reviewing, checking security

Posting comment to MR

Gitlab provides the way to get CI, just create .gitlab-ci.yml on root of the project directory.

Gitlab API

There is no special command to post comment within CI. We have to call REST API using curl.

Installing glab

I don’t like curl, so I asked GPT another way, he let me know the command ‘glab’

However it is not built-in command in Gitlab CI. We need to install it to use.

brew install glab

That’s all?

Authenticating glab

To run glab on your machine, it should be logon first.

glab auth login

So this prompt will be presented .

What Gitlab instance do you want to log into?

Select ‘Gitlab Self-hosted Instance’

Answer following questions.

  • Gitlab hostname: {gitlab domain}
  • API hostname: {gitlab domain}
  • How would you like to sign in? Token
  • Choose default Git protocol: HTTPS
  • Authenticate Git with your GitLab credentials? Y / N
  • Choose host API protocol: HTTPS

Note: the hostname should be domain not any name(your company?).

Posting note on MR

We can put comments on MR using the following Command line.

glab mr note {MR ID} -R {repository path} -m {comment}

Git Diff

My final purpose to post comment from AI about Changes. To get it, we should send the result of git diff to AI.

Branch Names

To run git diff on CI, I need to obtain source and target branch name.

Gitlab provides environment variables for that. However my CI is not on main branch yet, therefore I can’t use $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME.

So I defined it with current branch name $CI_COMMIT_BRANCH

script:
    # Define Additional Variables
    - CI_MERGE_REQUEST_SOURCE_BRANCH_NAME=$CI_COMMIT_BRANCH

and also target branch manually.

variables:
    CI_MERGE_REQUEST_TARGET_BRANCH_NAME: "releases/v6.0.15"

Fetching Branches

I don’t know exactly, however the runner would have old branch information.

So I fetched them before executing ‘git diff’.

- git fetch origin "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"
- git fetch origin "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME"

Posting Changes

We can get changes from git with this command now.

git diff “$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME” “$CI_MERGE_REQUEST_TARGET_BRANCH_NAME”

Let’s Post this changes as a comment on MR

- GIT_DIFF = $(git diff "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME" "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME")
- |
  glab mr note "$CI_MERGE_REQUEST_IID" -m "This is MR Comment Test. 
      Source[$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME]  
      Dest[$CI_MERGE_REQUEST_TARGET_BRANCH_NAME]  

      git diff result  
      $GIT_DIFF" -R "$GITLAB_HOST/${WORKSPACE}/${PROJECT}"

$CI_MERGE_REQUEST_IID would be MR ID like this

In this case, MR ID is 229.

Ollama

Now we have git changes. Final step is to ask AI to review it.
If you already have another AI solution, just use it.
Otherwise and if you don’t want to send money, use Ollama.

Installing Ollama

It is very simple to install Ollama.

Running Ollama is also very easy.

ollama run {model name}

However where can we get model names? Visite here.

You should keep in mind that large models are smarter, but it would make also slow down your machine and reduce available spaces.

Running AI on the machine with the runner

We can run Ollama with just one prompt.

ollama run {model} “{prompt}”

If the prompt is very large, we can give through a file.

ollama run {model} < prompt_file

This approach is simple, however each prompt will launch a new Ollama instance.

So I tried to find a way to share the one instance.

Ollama is also a service, it means we can ask him through Web API like this

curl -X POST http://localhost:11434/api/generate \
-H "Content-Type: application/json" \
-d '{"model": "{model}", "prompt": "{prompt}", "stream": false}'

API Domain is localhost and port is 11434. (I don’t know why 11434 and where I can change it)

Put application/json as Content-Type, because we want to get the result as JSON.

Request Body is also JSON. Let’s look it in detail.

{
    "model": "{model}", 
    "prompt": "{prompt}", 
    "stream": false
}

We already know what model and prompt are, I guess stream would return the result like human(ChatGPT) writing.

So Ollama emit the response as JSON

{
    "model":"qwen2.5-coder:7b",
    "created_at":"2025-03-10T05:06:59.263685Z",
    "response":"2+2 equals 4.",
    "done":true,
    "done_reason":"stop",
    "context":[151644,8948,198,2610,525,1207,16948,11,3465,553,54364,14817,13,1446,525,264,10950,17847,13,151645,198,151644,872,198,3838,374,220,17,10,17,30,151645,198,151644,77091,198,17,10,17,16819,220,19,13],
    "total_duration":11079613541,
    "load_duration":581527250,
    "prompt_eval_count":36,
    "prompt_eval_duration":10126000000,
    "eval_count":8,
    "eval_duration":370000000
}

Keep in mind the ‘response’ field is most important to our purpose. Ignore else.

Source Review

Now we are ready to ask to AI using the command.

Let’s give the result of ‘git diff’ to AI.

As shown above, I used curl to communicate with AI. However the result of ‘git diff’ would be very large, so it couldn’t work.

To avoid long curl command, we can specify the file for body of REST API like this.

curl -X POST {REST API url} \
… \
-- data-binary @{Body File Name}

So curl will use the file as the HTTPbody.

I created the file containing the result of ‘git diff’

git diff “$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME” “$CI_MERGE_REQUEST_TARGET_BRANCH_NAME” > git_mr_diff

Invalid JSON

However the curl body file should be JSON format, so we can’t use git_mr_diff directly to the body.

As we see above, the request JSON should be like follows.

{
    "model": "{model}", 
    "prompt": "Please review this git diff. {Content of git_mr_diff}",
    "stream": false
}

This JSON would not work due to invalid format.
git_mr_diff may have line feeds or double quotes and they will break JSON.

You would encounter this error.

bash: line 163: /usr/bin/curl: Argument list too long

To fix this problem, we need a tool jq, Command-line JSON processor.

Let’s Install jq.

brew install jq

The command ‘jq -Rs’ can fix the content for JSON.

git diff “$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME” “$CI_MERGE_REQUEST_TARGET_BRANCH_NAME” | jq -Rs > ollama_json

This command will attach double quotes like ‘“{git diff}”’, not ‘{git diff}’. Because it creates JSON field content.

"{\"model\": \"{model}\", \"prompt\": \"Please review this git diff. {Content of git_mr_diff}\",\"stream\": false}"

So we need to run jq with the entire prompt.

- echo -n ‘{“model”:”qwen2.5-coder:7b”,”prompt”:’ > ollama_json
- echo -n ‘Review GitLab MR (git diff, under 100 lines):please review git diff ’$(git diff “origin/$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME” “origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME”) | jq -Rs >> ollama_json
- echo -n ‘,”stream”:false}’ >> ollama_json

The first line would write the content previous to value of the prompt field.

{"model": "{model}", "prompt":

Second puts JSON qualified git diff into the file.

"Please review this git diff. {Content of git_mr_diff}\"

Last line finishes JSON format

,\"stream\": false}"

Extracting only response

If you run glab with response of AI, the system will emits error.

../bin/glab: Argument list too long

This error is caused by long length ‘context’ field of JSON from Ollama.

As I mentioned before, what we need only from AI is ‘response’, he’s answer.

To extract the specific field of JSON via command. Use -r options of jq.

jq -r “{field name}”

We can create a variable for AI response using this command.

AI_REVIEW=$(curl -X POST http://localhost:11434/api/generate \
      -H "Content-Type: application/json" \
      --data-binary @ollama_json | jq -r '.response')

Now Ollama can review my git changes like following.

- | 
  glab mr note "$CI_MERGE_REQUEST_IID" -m "MR Review 테스트 입니다.  
  Source[$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME]  
  Dest[$CI_MERGE_REQUEST_TARGET_BRANCH_NAME]  

  # AI 소스 검토 결과  
  $AI_REVIEW" -R "$GITLAB_HOST/{Team}/{Project}"

Maybe you should tune the prompt to concise AI review.

Merge Request Only

This CI would be run whenever pushing. As I mentioned in the title, I wanted to run this CI for the MR.

To restrict running it, we need to set the condition if run it.

rules:
    # Trigger only on MR creation (GitLab 14.5+)
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"
    - when: never

If you want to run CI only for MR Creation, we can add this option.

&& $CI_MERGE_REQUEST_EVENT_TYPE == "merge_request_created"'

See more variables for Merge Request.

However this works only when master branch has this CI.
So you should merge it into master first.

And you can see this when you create a new merge request 🎉 🙌🏻

Commenting on the source lines

I expected AI to put comment like human. Therefore I searched ways to put comment on the code line.

But glab doesn’t support like the option according to their documentation.

However I didn’t give up and attempted to use API directly.

According to the API doc, we can create threads on changes of merge request through API.

Body

curl --request POST \
  --header "PRIVATE-TOKEN: <your_access_token>"\
  --form 'position[position_type]=text'\
  --form 'position[base_sha]=<use base_commit_sha from the versions response>'\
  --form 'position[head_sha]=<use head_commit_sha from the versions response>'\
  --form 'position[start_sha]=<use start_commit_sha from the versions response>'\
  --form 'position[new_path]=<new file path>'\
  --form 'position[old_path]=<old file path>'\
  --form 'position[new_line]=<line number to put comment>'\
  --form 'body=<comment message>'\
  --url "https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions"

You can use the project ID instead of {Space Name}%2F{project name} if you know.

It seems like that I have to modify this file to do that using glab

commands/mr/note/mr_note_create.go · main · GitLab.org / cli · GitLab

A GitLab CLI tool bringing GitLab to your command line

gitlab.com

or I need to implement new mr_discussion_create.go.

Conclusion

Don’t give up even if your development environment is prevented from using AI Services due to security. Construct your one private AI Service. It is secure and there is no payment requirement.

Try to use new technology for your workflow to save your time.

I will be back after achieving line comment with glab. 😎

If you found this post helpful, please give it a round of applause 👏. Explore more AI-related content in my other posts.

For additional insights and updates, check out my LinkedIn profile. Thank you for your support!

References

Predefined CI/CD variables reference

GitLab product documentation.

docs.gitlab.com

Discussions API

GitLab product documentation.

docs.gitlab.com

AI
Gitlab
Ollama
Ci
Review

--

--

Lee young-jun
Lee young-jun

Written by Lee young-jun

107 followers
7 following

Man

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Rules

Terms

Text to speech