Niceness: Reduce the priority of the local and/or remote rsync process by command line option similar to nice or ionice

[michaelmess-de]

Problem:

rsync with plenty of files and data (e. g. for backup purposes) puts load on the client and server machine and people working on these machines might experience a noticeable performance degradation when hardware comes to it's limits. As a workaround the user may renice or ionice the rsync processes to mitigate the issue, but this requires ownership of the running rsync-processes or root access and only works temporarily on the already running process.
Thus in several cases it would be desireable that rsync (e. g. when automatically started by a script or cron job, etc.) gives way for other activities, reducing the impact on those working on the machines.

Solution:

Added command line options to lower the priority of the rsync process like with renice (lower CPU priority) or ionice (lower IO priority), so that other activities on the machine are not affected by the running rsync job so much.

--nice-local             renice local process to low priority
--nice-remote            renice remote process to low priority
--ionice-local           ionice local process to low priority
--ionice-remote          ionice remote process to low priority
--nice, --ionice         set local and remote process to low priority
--no-nice, ...           prepending "no-" turns the given option off
--nice-and-ionice, -Q    same as --nice --ionice

The option names are derived from the well known unix tools nice/renice and ionice that can be used to alter priorities of running processes.
With --nice-local or --ionice-local, if there is no support on the local machine OS for reducing priority, despite of a current rsync version, the option is silently ignored.
With --nice-remote or --ionice-remote, if there is no support on the remote machine OS for reducing priority, despite of a current rsync version, the option is silently ignored. If in this case the rsync version on the remote machine does not yet support the new command line options, rsync reports an error.

[cdlscpmv]

How is it different from prepending nice and/or ionice commands to rsync or --rsync-path (in the remote case)?

[michaelmess-de]

How is it different from prepending nice and/or ionice commands to rsync or --rsync-path (in the remote case)?

Prepending nice and/or ionice to the remote depends on that these utilities have actually been installed by the user. We do not know, if this is the case on the remote site and if not, the remote command will fail to execute rsync.
But on the other hand, we also do not know, if rsync on the remote site already does know about the option (It might need an upgrade to a current version that supports it). Thus there is currently only little difference. But in the future, when systems have been upgraded, the new options should be known by the remote end, and, if the feature is actually not supported by the remote OS, this would just issue a warning and would not stop us from rsyncing successfully.

With prepending nice/ionice there would be also additional processes to be launched, maybe that could be an issue.
Also when starting nice and ionice remotely, we rely on other software on the remote shells path, which could also be security relevant. And in restricted environments where only rsync is made available as executable, this would definitely fail.
With the feature integrated in rsync we just tell the kernel to reduce the priority directly (if supported on the architecture where the binary was compiled for), this should be safe and efficient.
And in the local case, the user/script has already started rsync, maybe with nice/ionice, maybe not.
Adding an option to rsync is much easier for the user and thus more reliable and less error-prone than fiddling with nice/ionice on the command line.

[cdlscpmv]

Prepending nice and/or ionice to the remote depends on that these utilities have actually been installed by the user. We do not know, if this is the case on the remote site and if not, the remote command will fail to execute rsync.

--rsync-path is a shell command, so you can include tests like $(which nice >/dev/null 2>/dev/null && echo nice -n 10) rsync and similarly for ionice.

And in restricted environments where only rsync is made available as executable, this would definitely fail.

The administrator of the restricted environment can be contacted to provide the means to resolve the issue (if they see it as an issue at all, because they may not be concerned about high CPU/disk usage).

[michaelmess-de]

Thank you for the workaround with --rsync-path. It is a good workaround for the meantime and works fine with linux.
But most people like it easier.
And maybe the user account could be configured to use another shell, maybe on Windows PowerShell. That could break the --rsync-path workaround or make it more complicated to handle all cases.
Thus it can help some people, but is not a general solution that could be rolled out e.g. in combination with BackupPC or other tools.

[devZer0]

thanks for making an enhancement to rsync.

I'm a little bit surprised, that we have that many new options for adjusting niceness, but without the capability of the original tools, i.e. if you add this complexity, i think adjusting the level to the desired values should also be possible.

anyhow, i'm not sure if that belongs into rsync, i don't have yet an opinion on that.
but when it is integrated, i strongly vote for adding levels, too

ionice has:

ionice [-c class] [-n level] [

-c, --class class
           Specify the name or number of the scheduling class to use; 0 for none, 1 for realtime, 2 for best-effort, 3 for idle.

-n, --classdata level
           Specify the scheduling class data. This only has an effect if the class accepts an argument. For realtime and best-effort, 0-7 are valid data (priority levels), and 0 represents the highest priority level.
	

nice has:

-n, --adjustment=N

add integer N to the niceness (default 10)
						 
Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process).

[michaelmess-de]

I got the idea, that this might be useful, when I was working on something and felt then the computer's performance went poor after a BackupPC job started.
As a workaround, I used to type "sudo renice 20 ;sudo ionice -c3 -p" to get back to performance, but it is always work to do that.
I just wanted to achieve that the rsync process went to the background, not taking any noticeable amount of performance, and I came up with the idea that this would be a very useful feature to be integrated into rsync.
Then I found, this would be much more complex than expected. There are 2 rsync processes, local and remote and there are 2 tools, renice (for CPU usage) and ionice (for disk usage).
The goal was to make it as simple to use as possible, and when analysing the parameter processing I found that I had to implement the no-... options, too, just to be consistent, which resulted in 8 options to switch something on or off.

I thought about implementing levels already, but found the following issues:
For Renice: Usually we want to lower priority to reduce the performance impact on other activities, increasing priorities require special permissions (root or CAP_SYS_NICE) and would even increase the impact on other activities on the system. Specifying levels would make sense if there were plenty of processes on the system and there is the need to give them different priorities.
If there is a high CPU load, renice would always be beneficial to give way for other activities, the effect would be the biggest, if prio ist lowest.
If there is no CPU load at all, rsync won't suffer from a low prio given.
From my experience I thought, there won't be the need to give special levels to rsync and thus I left that out.

For Ionice: We have four classes: 0 and 2 both are effectively best-effort, 1 is real time and 3 is idle. In addition to that, there are seven priorities that can be set in addition to the class and are only effective for class 1 and 2. For class 0 the priority is taken from the nice value. When we want to bring heavy io load of rsync into the background, usually 3 (idle) is the best option, best effort with other priorities brings only a small, negligible advantage. Running rsync in real-time is possible and needs root, but does it actually make sense?
Experience varies with the different disk types, e. g. a hard disk can be slowed down by random access while an ssd does not have this issue. On a hard disk the idle class really gives an improvement, while best effort won't. Maybe on an ssd experience might be different and might be similar to renice, but I haven't tested that yet.
Thus I have thought, that other values than IDLE won't be useful for ionice and left other values out, just to keep it simple.

But it would be easy for me to change the functionality of the renice switch to contain a renice level instead of just a boolean switch and add another option which takes an argument for the nice priority that is then also used to pass the priority to the remote rsync. The existing options will remain unchanged and renice to 19 when turning nice on. If a level was already set there, it shouldn't be overwritten by turning it on.
For ionice this would be a bit more complicated due to the valid combinations of class and prio, but that should also be possible.

I think it is a good idea for me to develop this on a new branch derived from this, so that the current state can already be reviewed and merged and thus already made available while I am working on the improvements on another branch.

I will add two voting comments (vote by adding a like) for the mentioned improvements.

[michaelmess-de]

VOTE: Like to vote for specifying nice levels in rsync

[michaelmess-de]

VOTE: Like to vote for specifying IOnice levels in rsync

[cdlscpmv]

ionice is platform-specific, so adding it to rsync is not a good idea.