How To Create CentOS 8 Local Repository Mirrors With Rsync & Nginx

0
249

Today, we are going to create CentOS 8 Local Repository Mirrors on our CentOS 8 with the help of Rsync & Nginx. We can create by Local cron jobs through which we can checks for updates in the upstream repositories & update them locally accordingly.

By doing so, we’ll be able to configure our CentOS 8 servers to access the packages without connecting to the Internet. It helps us in reducing the amount of BandWidth and improves security control. Time can also be saved which is being consumed by pulling RPM packages from external sources.

Before going to start make sure you’ve an external storage with enough capacity to host repository packages & potential growth.

Step 1: Install Nginx Server

sudo dnf -y install @nginx

Start & enable the nginx services.

sudo systemctl enable --now nginx

Double check that the status is in working state.

systemctl status nginx

Allow Firewall control.

sudo firewall-cmd --add-service=http --permanent
sudo firewall-cmd --reload

Step 2: Create Repository Directories

In my server the secondary storage capacity is of 100 GB that I can easily use to store data /dev/vdc.

$ lsblk 
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda 252:0 0 20G 0 disk
├─vda1 252:1 0 1G 0 part /boot
└─vda2 252:2 0 9G 0 part
├─cl-root 253:0 0 8G 0 lvm /
└─cl-swap 253:1 0 1G 0 lvm [SWAP]
vdc 252:32 0 100G 0 disk

Now, we will create partition on this disk and mount it under /data directory.

sudo parted -s -a optimal -- /dev/vdc mklabel gpt
sudo parted -s -a optimal -- /dev/vdc mkpart primary 0% 100%
sudo parted -s -- /dev/vdc align-check optimal 1

Next, create a physical volume & group volume in this disk.

$ sudo pvcreate /dev/vdc1
Physical volume "/dev/vdc1" successfully created.

$ sudo vgcreate data /dev/vdc1
Volume group "data" successfully created

In the next step we’ll create a logical volume on this disk

$ sudo lvcreate -n repos -l+100%FREE data
Logical volume "repos" created.

Now, create a file system on the given disk.

$ sudo mkfs.xfs /dev/mapper/data-repos
meta-data=/dev/mapper/data-repos isize=512 agcount=4, agsize=1965824 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=1
data = bsize=4096 blocks=7863296, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=3839, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0

To mount the Logical Volume, make a directory for it.

sudo mkdir /data

Now, create a mount point

sudo vim /etc/fstab
/dev/mapper/data-repos /data xfs defaults 0 0

Proceed with mounting it.

sudo mount -a

Make sure it is mounted.

$ df -hT /data
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/data-repos xfs 100G 247M 100G 1% /data

For storing data in repositories, create a base directory.

sudo mkdir -p /data/repos/centos/8/

Make sure all directories are created.

$ tree /data/
/data/
└── repos
└── centos
└── 8
3 directories, 0 files

Step 3: Create Repositories Sync script

We need a script for syncing contents from the remote repositories to the our local system

sudo vim /etc/centos8_reposync.sh

Copy and paste the given data into that file.

!/bin/bash
repos_base_dir="/data/repos/centos/8/"
Start sync if base repo directory exist
if [[ -d "$repos_base_dir" ]] ; then
# Start Sync
rsync -avSHP --delete rsync://mirror.liquidtelecom.com/centos/8/ "$repos_base_dir"
Download CentOS 8 repository key
wget -P $repos_base_dir wget https://www.centos.org/keys/RPM-GPG-KEY-CentOS-Official

Run the following command to make the script executable.

sudo chmod +x /etc/centos8_reposync.sh

Now install Tmux & initiate first execution.

sudo yum -y install tmux
tmux
sudo /etc/centos8_reposync.sh

You’ll see the script start running.

Various directories will be created.

$ ls -1 /data/repos/centos/8/
AppStream
BaseOS
centosplus
COMPOSE_ID
cr
extras
fasttrack
isos
PowerTools

When the script finish up then create a cron job for weekly sync.

You can set your desired time I’m going to set it every Sunday at 6 pm.

$ sudo crontab -e
00 18 * * 7 /etc/centos8_reposync.sh

Step 4: Configuring Nginx

First of all create a nginx config file to serve the repository contents.

$ sudo vim /etc/nginx/conf.d/centos.conf
server {
listen 80;
server_name repos.example.com;
root /data/repos/;
location / { autoindex on; }
}

Note: repos.example.com is the your domain name, so set it accordingly to access the repositories.

Make sure you’ve configure SELinux labels.

sudo semanage fcontext -a -t httpd_sys_content_t "/data/repos(/.*)?"
sudo restorecon -Rv /data/repos

Restart nginx services

sudo nginx -t
sudo systemctl restart nginx

Check that nginx is working by the URL repos.osradar.com
Note: You can use your own domain to access this.

Step 5: Configuring CentOS 8 Client machines

To use the local repositories, we have to configure our CentOS machines as our mirror is ready.

Take a backup of current repositories before proceeding.

cd /etc/yum.repos.d/
sudo mkdir old-repos
sudo mv *.repo old-repos

Make a new repository file.

sudo vim local.repo

Copy & paste the given data into above file.

[BaseOS]
name=CentOS-$releasever - Base
baseurl=http://repos.osradar.com/centos/$releasever/BaseOS/$basearch/os/
gpgcheck=1
enabled=1
gpgkey=http://repos.osradar.com/centos/$releasever/RPM-GPG-KEY-CentOS-Official
[AppStream]
name=CentOS-$releasever - AppStream
baseurl=http://repos.osradar.com/centos/$releasever/AppStream/$basearch/os/
gpgcheck=1
enabled=1
gpgkey=http://repos.osradar.com/centos/$releasever/RPM-GPG-KEY-CentOS-Official
[centosplus]
name=CentOS-$releasever - Plus
baseurl=http://repos.cosradar.com/centos/$releasever/centosplus/$basearch/os/
gpgcheck=1
enabled=0
gpgkey=http://repos.osradar.com/centos/$releasever/RPM-GPG-KEY-CentOS-Official
[extras]
name=CentOS-$releasever - Extras
baseurl=http://repos.osradar.com/centos/$releasever/extras/$basearch/os/
gpgcheck=1
enabled=1
gpgkey=http://repos.osradar.com/centos/$releasever/RPM-GPG-KEY-CentOS-Official
[PowerTools]
name=CentOS-$releasever - PowerTools
baseurl=http://repos.osradar.com/centos/$releasever/PowerTools/$basearch/os/
gpgcheck=1
enabled=0
gpgkey=http://repos.osradar.com/centos/$releasever/RPM-GPG-KEY-CentOS-Official
[cr]
name=CentOS-$releasever - cr
baseurl=http://repos.osradar.com/centos/$releasever/cr/$basearch/os/
gpgcheck=1
enabled=0
gpgkey=http://repos.osradar.com/centos/$releasever/RPM-GPG-KEY-CentOS-Official
[fasttrack]
name=CentOS-$releasever - fasttrack
baseurl=http://repos.osradar.com/centos/$releasever/fasttrack/$basearch/os/
gpgcheck=1
enabled=0
gpgkey=http://repos.osradar.com/centos/$releasever/RPM-GPG-KEY-CentOS-Official

Note: You can use your own domain for this.

Now clean up current repo cache by the following command

sudo yum clean all

After it update it.

$ sudo yum makecache
CentOS-8 - AppStream 1.2 MB/s | 4.3 kB 00:00
CentOS-8 - Base 3.9 kB/s | 3.9 kB 00:01
CentOS-8 - Extras 881 kB/s | 1.5 kB 00:00
Metadata cache created.

Now list repositories and run system update.

sudo yum repolist
sudo yum -y update

Step 6: Enable repositories which are being disabled

By default, some repositories are disable. Enable them by following command.

sudo yum istall yum-utils

Type given command to activate the repository.

sudo yum-config-manager --enable reponame

Enable PowerTools & centosplus repositories.

sudo yum-config-manager --enable PowerTools
sudo yum-config-manager --enable centosplus

Double check these

$ sudo yum repolist 
CentOS-8 - AppStream 1.7 MB/s | 4.3 kB 00:00
CentOS-8 - Base 2.0 MB/s | 3.9 kB 00:00
CentOS-8 - PowerTools 75 MB/s | 1.8 MB 00:00
CentOS-8 - Plus 58 MB/s | 833 kB 00:00
CentOS-8 - Extras 783 kB/s | 1.5 kB 00:00
repo id repo name status
AppStream CentOS-8 - AppStream 5,089
BaseOS CentOS-8 - Base 2,843
PowerTools CentOS-8 - PowerTools 1,507
centosplus CentOS-8 - Plus 26
extras CentOS-8 - Extras 3

To disable, substitue -enable with -disable.

Congratulations you’ve done. Here’s the similar tutorial if you want to sync through satellite.

How To Sync CentOS 8 repositories on Satellite / Katello / Foreman

LEAVE A REPLY

Please enter your comment!
Please enter your name here