Harbor is an open source registry as the trusted cloud native repository for Kubernetes.
First of all, Harbor is written in Go Lang, the web UI is nice looking, accessible and easy to use and fortunately, it has a rich and excellent RESTful API.
During one of our DevSecOps initiative we are required to integrate Harbor as the registry of our CICD pipeline and therefore, we jump into studying how to auto various tasks that can be done in Web UI and realized that REST API although is very powerful, it has some challenges in scripting every single request as REST API.
After talked to the product team, various reason of this is not going to happen soon enough (we want all feature ready yesterday, isn’t it?)
Thus this drives the born of the Harbor CLI, and CNCF always welcome contributions:
This is not yet an offical project for Harbor open source Registry Download the cli jar file directly in github: Github…
This blog will details the design principles and choices for the command line.
Harbor-cli is built-on?
Harbor-cli itself is written in Java using Spring Boot, a cross-platform way to allow the command line to run in various OS and CICD environment; Spring Boot brings neatness and useful utilities for dynamic discovery of available subcommands.
To avoid reinventing the wheel PicoCLI is used for user interaction and interface for scripting.
How Harbor CLI is structured?
We structure it into hierarchy of Resource -> Action -> Flags and parameters
In the above help message, the “scan”/”artifact”/”preheat” are all resources record you can config in Harbor.
You can see
harbor repository has actions of
get/update/list/delete , and below the individual action, there are specific parameters inside each action:
A big problem here: There are more than 120 API and numerous parameters, thus with over 1000 combination of resource + action + parameters, so how to make the command align with all API correctly and constantly updated in a smarter way?
How it’s made:
10,000ft High level view of life cycle of a CLI call:
Before we dive deep into how it works, here is the overall process of how the CLI works and dependencies:
First of all, let’s jump into how the CLI itself knows all the endpoints available from Harbor.
Dealing with RESTful API endpoints:
Building a CLI can be complex, there are 7 major API endpoint with >150 path to interact with Harbor, you can take a look at the sea of endpoints here: https://github.com/goharbor/harbor/blob/master/api/v2.0/swagger.yaml
Thus, there are 2 options:
- Manually code one by one
- Auto-generation and dynamic discovery of which API endpoint and parameters are there.
Manually coding, although usually yield a higher quality and tailor made with type safety, it will be time-consuming and repetitive, also when upstream Harbor project updated,
harbor-cli will not be able to keep up with or risk of re-do large portion of codes.
Auto-generation and dynamically create command line option, is complex and challenging but avoid repeat chores of fighting each command one-by-one, since Harbor is an active and constantly updating software, this method allows fast response to upstream changes, even breaking changes to upstream RESTful API. We only have to perform minor update to the dependencies of
harbor-client-java . The drawback is some corner cases that could not be handled in a user friendly way.
Manually vs Auto:
As you have guessed right, the choice has been made to cover most functions instead of highly-specific but no so complete one using auto-generation.
How auto-generation works?
harbor-cli is written in Java with Spring, we can include the
swagger.json, parse it, and generate tons of
RestTemplate and map it as subcommand.
harbor create project — >
restTemplate.exchange(url, HttpMethod.POST, request, ProjectReq.class);
A lot of moving parts here —Create URL, request and parsing responses etc etc for making a single call, which means driving backward again.
Therefore, an alternative method has been used: To generate a native Java Client that talks to API server.
We clone the swagger.json from upstream goharbor repo, then our pipeline (which runs in GitHub), generate an artifact which published back to GitHub repo here:
Java Client for Harbor open source trusted cloud native registry https://github.com/goharbor/harbor Go to release page…
We generate a usable
harbor-client-java using OpenAPI generator, and compile to a single Jar file, the process has been exactly documented in my another blog: How Kubernetes support so many client libraries?
What do we get from this `harbor-client-java` process?
A JAR file with:
- All the API endpoints to chat with the server (eg: ArtifactAPI); and
- All the data model (eg: Artifact) that you need to build/parse from server side
Next, we include this JAR file into our main CLI
Now we transform the problem from “How to talk to the Harbor API” to “Our CLI just need to call the right Java class”. No more
RestTemplate , no more direct HTTP calls needed to be taken care by our
cli , as we have delegate it to classes in
But how do we show the API to the user?
harbor-cli just need to use Java’s reflection api to retrieve the
apis and add to the list of top level command:
One exception is: Both
logout are manually written to specify how credentials are saved and loaded to make the
harbor-cli usable in CICD script.
How about sub-commands and command’s help message?
Again, Java reflection, we leverage the fact that
calling class method == calling API endpoint , here are the steps:
- Query a list of class methods inside
- Run a filter to remove irrelevant methods
- Finally add to the sub-command list
Similar to sub-command, all the options are added using Java reflection:
How about data send to API server?
ProjectReq is some complex Java data model:
That’s why you should write the data model separately, in a Json file to describe what’s
ProjectReq and example here:
Then you can use the
harbor project create --project=path_to/myprojectspec.json
What actually happen when the user press enter?
We do the reverse!
We know the API class name, action and parameters will be in the command line, therefore, we do the mapping and again, using Java’s reflection API to
invoke the correct Java API.
Then we just print the result or in the case of error, we print the exception, straightforward.
Once the user/script got the response, it is a nice JSON and using
jq command, you can programmatically parse the part that you wanted.
harbor-legacy shared the same code base with
harbor cli, Harbor project is updating it’s RESTful API, so it is in the middle of having 2 sets of API endpoints.
harbor-legacy contains tons of admin related API such as creating users, updating global system config etc:
So if you are considering automating and setting up a reproducible or immutable harbor system config, you can use
After chatting with the
Harbor project team members, there is a plan for gradually move everything to the newer API endpoints and thus
harbor-legacy will follow the upstream
legacy-swagger fade out schedule.
One can view
harbor-cli as the wrapper for
harbor-client-java which is also a wrapper for RESTful API defined in OpenAPI specification, by leveraging this ring of automation, we can design and implement a usable CLI with benefits of lock step update from upstream changes, a scriptable interface for CICD pipeline and address use cases that you can’t and don’t want to interact with raw HTTP RESTful requests.
Try it NOW!
The Harbor CLI is usable and available at GitHub release page. Battle testing it in real world!
- We want more contributors! Pull request needed to make it elegant, feature rich and constant bug fixes!
- Generalization of this method of so that we can wrapping any OpenAPI Clients to CLI could be benefiting
If you have any question, please feel free to contact me in GitHub!
Version history: 1-Feb-2021 Added command structuring and