15 Examples to Upload Chef Repo with Cookbooks Using Knife Command

Once you’ve created your recipes and other objects for your chef cookbook, it should be uploaded to the Chef server.

There are two ways you can upload your cookbooks:

1. Using knife cookbook sub-command – This helps you to manipulate your cookbooks
2. Using knife upload sub-command – This helps you to upload all chef objects (including cookbooks, environments, nodes, etc.)

This tutorial explains how to use both of these sub-commands (knife cookbook and knife upload) with examples.

The following are covered in this tutorial:

1. Upload Single Cookbook
2. Upload ALL Cookbooks
3. Freeze Cookbook Version (and Force Upload)
4. Upload Cookbook with All Dependencies
5. Increase Number of Concurrent Connections
6. Specify Cookbook Directory Location
7. Upload cookbooks using upload sub-command
8. Upload ALL Cookbooks (using upload sub-command)
9. Upload Roles, Nodes and Environments
10. Upload the Whole Chef Repo (or) Upload Selective Items
11. Force Upload Even When It’s not Changed Locally
12. Perform Dry Run to Verify What’ll get Uploaded
13. Recursive and Concurrency Options
14. Chef Repo Mode Option
15. Delete Cookbook from Chef Server

I. Knife Cookbook Sub-command

1. Upload Single Cookbook

Using the knife cookbook sub-command, you can perform all kinds of operations on your cookbooks including create, delete, upload, lock, view, etc.

To create a new Chef cookbook, refer to this: 12 Chef Knife Cookbook Command Examples

To upload a single cookbook, use the following syntax:

knife cookbook upload [cookbook-name]

Using knife cookbook sub-command, the following will upload a cookbook to the Chef server. The following will upload the cookbook called thegeekstuff.

Before you upload your cookbook, make sure you working directory is your local chef repository directory. In this example, it is ~/chef-repo. The following uploads prod-db cookbook.

# cd ~/chef-repo

# knife cookbook upload prod-db
Uploading prod-db [1.0.0]
Uploaded 1 cookbook.

Note: When you do an cookbook upload, only the files that doesn’t already exist in the chef server will be upload. So, this is kind of an incremental upload.

2. Upload ALL Cookbooks

If you want to upload multiple cookbooks, use the -a option as shown below.

The following will upload all the available cookbooks on your local chef workstation from the current directory.

The following command will upload all your local cookbooks to chef server.

# knife cookbook upload -a
Uploading prod-db        [1.0.0]
Uploading thegeekstuff   [2.1.0]
Uploaded all cookbooks.

Note: If you don’t want to upload all the files from the cookbooks, you can use the chefignore file to exclude certain file types. For example, you can exclude .git files (or other temporary files) by specifying them in the chefignore file.

You can also use –all instead of -a option.

knife cookbook upload --all

If you are execute the upload command outside your chef-repo directory, you’ll get the following ” Your private key could not be loaded from /etc/chef/client.pem” ERROR message.

# knife cookbook upload -a
WARNING: No knife configuration file found
WARN: Failed to read the private key /etc/chef/client.pem: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /etc/chef/client.pem>
ERROR: Your private key could not be loaded from /etc/chef/client.pem
Check your configuration file and ensure that your private key is readable

3. Freeze Cookbook Version (and Force Upload)

Once you’ve done all your work on a particular version of the cookbook, you can execute the following command, which will freeze that particular version.

# knife cookbook upload prod-db --freeze
Uploading prod-db        [1.0.0]
Uploaded 1 cookbook.

In this example, the 1.0.0 version of prod-db cookbook is frozen. This means that if you want to modify prod-db, you should create a new version of the cookbook, and upload the new version.

If you are trying to update a frozen cookbook, you’ll get the following error message:

# knife cookbook upload prod-db
Uploading prod-db        [1.0.0]
ERROR: Version 1.0.0 of cookbook prod-db is frozen. Use --force to override.
WARNING: Not updating version constraints for prod-db in the environment as the cookbook is frozen.
ERROR: Failed to upload 1 cookbook.

But, for some reason, if you want to update a fronzen cookbook, then use the –force option as shown below.

# knife cookbook upload prod-db --force
Uploading prod-db        [1.0.0]
Uploaded 1 cookbook.

Note: During a cookbook upload, if you have empty directory on your local machine, they’ll not be uploaded to the Chef server. If you want empty directories to be upload, add some hidden files like .empty inside the empty directory. In that case, technically that directory is not empty, and it will be uploaded by the knife cookbook upload command.

4. Upload Cookbook with All Dependencies

When you are uploading a single cookbook, you can also use -d option (or) –include-dependencies to upload all the dependent cookbooks automatically as shown below.

knife cookbook upload thegeekstuff -d

knife cookbook upload thegeekstuff --include-dependencies

5. Increase Number of Concurrent Connections

By default, the number of concurrent connections the knife command uses for uploading the cookbooks to the chef server is 10. This default value is good enough for most situation. But, if you are having 100s of large cookbooks, you can speed-up the upload process by increasing the concurrency value as shown below.

# knife cookbook upload -a --concurrency 20

6. Specify Cookbook Directory Location

You can also specify the location of your cookbook using the -o option. Here, you can specify more than one directory location as shown below. This will look for the cookbook to be uploaded in the specified directory.

# knife cookbook upload prod-db -o /root/chef-repo/cookbooks
Uploading prod-db      [1.1.0]
Uploaded 1 cookbook.

When specifying multiple cookbook directories, separate them with colon as shown below.

# knife cookbook upload prod-db -o /root/chef-repo/cookbooks:/home/chef/cookbooks

You can also use –cookbook-path, which is same as the -o option

# knife cookbook upload prod-db --cookbook-path /root/chef-repo/cookbooks

# knife cookbook upload prod-db --cookbook-path /root/chef-repo/cookbooks:/home/chef/cookbooks

II. Knife Upload Sub-command

Knife upload sub-command is used to upload all chef objects including cookbooks. For example, you can use this to upload cookbooks, roles, environment variables, etc.

7. Upload Cookbooks using Upload sub-command

To upload a specific cookbook, go to the chef-repo directory, specify the cookbook name along with the cookbook directory as shown below. This will upload prod-db cookbook from local machine to the Chef Server.

# knife upload cookbooks/prod-db
Updated cookbooks/prod-db

Please note that this will do the upload only if anything is changed in the cookbook locally.

8. Upload ALL Cookbooks

First, cd to the chef repository directory, and execute the following, which will upload all the cookbooks from your local machine.

# cd chef-repo

# knife upload cookbooks
Updated cookbooks/prod-db
Updated cookbooks/thegeekstuff

You can also upload cookbooks from anywhere inside the chef repository. For this, specify / in front of cookbooks directory name as shown below.

# knife upload /cookbooks
Updated cookbooks/prod-db
Updated cookbooks/thegeekstuff

In the above example, we are in the ~/chef-repo/nodes directory, and uploading all the cookbooks.

9. Upload Roles, Nodes and Environments

To upload the roles JSON files, execute the following:

knife upload roles

To upload the nodes JSON files, execute the following:

# knife upload nodes
Created nodes/webserver.json

To upload environments, do the following:

knife upload environments

For the roles/nodes/environments, just like cookbooks, you can specify / in front of the directory, and execute the following command from anywhere inside your local chef repository directory.

knife upload /roles

knife upload /nodes

knife upload /environments

Also, you can upload a specific JSON file for either roles or nodes or environments by specifying the name of the JSON file as shown below.

knife upload roles/dba.json

knife upload nodes/webserver.json

knife upload environment/oracledb.json

10. Upload the Whole Chef Repo (or) Upload Selective Items

To upload your entire local chef repository to your server, cd to the chef-repo directory and execute the following.

# knife upload . 
Updated nodes/webserver.json
Updated cookbooks/prod-db
Updated cookbooks/thegeekstuff
Updated roles/dba.json
..

The following is same as the above command, except this can be executed from anywhere inside your local chef repository directory.

knife upload /

Or, you selectively upload few object types. For example, the following will upload the cookbooks and nodes that match the given pattern.

# knife upload cookbooks/prod* nodes/web*
Updated nodes/webserver.json
Updated cookbooks/prod-db

11. Force Upload even when it’s not Changed Locally

Just like the cookbook sub-command, the upload sub-command also supports freeze and force option as shown below.

knife upload cookbooks/prod-db --freeze

Once a cookbook is frozen, you can upload the same version anymore as shown below.

# knife upload cookbooks/prod-db
ERROR: cookbooks failed to write: Cookbook prod-db is frozen

Once a particular version of a cookbook is frozen, for some reason, if you want to update that without creating a new version, use the –force as shown below.

# knife upload cookbooks/prod-db --force
Updated cookbooks/prod-db

Note: Please note that, –force command can be used not only to update a frozen version, but also to update all the files in the chef server, even when the file is not changed locally.

For example, the following updates only prod-db, as only that particular cookbook was changed locally.

# knife upload cookbooks
Updated cookbooks/prod-db

But, when you give –force, all the local cookbooks will be uploaded even if they are not changed locally.

# knife upload cookbooks --force
Updated cookbooks/prod-db
Updated cookbooks/thegeekstuff
..

12. Perform Dry Run to Verify what’ll get Uploaded

Dry run is very helpful if you really want to see what local cookbooks are changed, which will be uploaded to the chef server. This way, you can make sure the upload command will upload what you are really expecting it to upload.

The following shows that this would upload only the prod-db and webserver cookbook. Nothing is really uploaded. This is just a dry run.

# knife upload cookbooks --force --dry-run
Would update cookbooks/prod-db
Would update cookbooks/webserver

13. Recursive and Concurrency Options

By default, the knife upload command, does all the upload recursively. If you don’t want recursive upload, you can specify no-recurse as shown below.

knife upload cookbooks --no-recurse

By default the number of concurrent connections the knife upload command will use is 10. If you have 100s of objects in your local machine that should be uploaded to the chef server, you can increase the concurrent connections to upload as shown below.

knife upload cookbooks --concurrency 20

14. Chef Repo-Mode Option

The default repo mode used by the knife upload command is either everything or hosted_everything depending on how your chef server setup.

The following are the three possible value for chef repo mode:

1. static
2. everything
3. hosted_everything

You can also use static if you are using only cookbooks, roles, environments and data bags as shown below.

# knife upload cookbooks --repo-mode static
Updated cookbooks/thegeekstuff
Updated cookbooks/webserver
Updated cookbooks/prod-db

For your local chef server (Which is installed on your own server), if you try to use everything repo-mode, it will give the following error message:

# knife upload cookbooks --dry-run --repo-mode everything
WARNING: You have repo_mode set to 'everything', but your chef_server_url
         looks like it might be a hosted setup.  If this is the case please use
         hosted_everything or allow repo_mode to default

In this case, use the hosted_everything mode as shown below.

# knife upload cookbooks --repo-mode hosted_everything
Updated cookbooks/thegeekstuff
Updated cookbooks/webserver
Updated cookbooks/prod-db

15. Delete Cookbook from Chef Server

If you’ve removed a cookbook from your local chef workstation, it will still be there on your chef server.

First, remove a cookbook from your local server.

cd ~/chef-repo

rm -rf prod-db

Force upload all your local cookbooks

# knife upload cookbooks --force
Updated cookbooks/thegeekstuff
Updated cookbooks/webserver

But, if you view the cookbooks in the Chef server, you’ll still see the erased prod-db cookbook.

# knife cookbook list
thegeekstuff    0.2.0
webserver       0.1.0
prod-db         0.1.0

So, to permanently delete it from the Chef server, use the –purge option as shown below.

# knife upload cookbooks --purge
Deleted extra entry cookbooks/prod-db (purge is on)

After the purge, we will not see the prod-db cookbook on the remote server anymore as shown below.

# knife cookbook list
thegeekstuff    0.2.0
webserver       0.1.0