Prefetching in Linux?

[Debian_SuperUser]

Sometimes when I am done with heavy web browsing, the system is not as fast as a fresh reboot. One thing is that a lot of the memory has been paged, but I can just disable and re-enable the swap space to load all the paged memory back in RAM. But the other thing is a lot of my cache is gone, and same thing after boot of course. I have to manually open my programs for them to be cached. Just like Sysmain in Windows, is there a prefetcher for Linux? I tried preload, but that doesn't seem to be doing much.

 

[Debian_SuperUser]

@osprey

What?? Why does my specs matter at all? I just wanted to know if there is something similar to Sysmain on Linux which can prefetch data as cache.

 

[Aristarchus]

Maybe
swappiness
vfs_cach_pressure
need to be adjusted.
Setting swappiness high will tell kernel to move all that is not needed to swap, freeing RAM for caching files. When you set vfs_cache_pressure low (e.g 50 but not 0), it will favor caching files instead of keeping application data in RAM.
There are other VM parameters that you can try too.
Also, what you do does not make much sense. Seems that you don't have enough RAM. Maybe try to configure ZRAM (if not not hibernating system, you can disable swap file/partition) or ZSWAP (if you need hibernation) . Both are faster than classic swapping. Also use lz4 as compressor.

In the end your system specs matter. Does not matter what OS.

 

[bob466]

Prefetch...now there's a word I haven't heard for a long time....you might want to look at this...
https://phoenixnap.com/kb/swappiness

 

[GatorsFan]

Sometimes when I am done with heavy web browsing, the system is not as fast as a fresh reboot.

This is caused by the cache filling up requiring more RAM to sift through it - If you use Firefox you can have it clean that at web browser shut down - Settings>Privacy&Security>Cookies & Site Data ensure a check mark is placed in "Delete Cookies and Site data" next scroll down to History and select from the Drop Down menu "Never Remember History" this will clean your internet cache

I always keep my cache clean it slows down browsing profiling since every-time you open up your browser things like google analytics has to start over on it's profiling your sites that you visited

Myself I use BleachBit to clean the cache and temp files - just make sure you set this up correctly or you can make a mess out of your system - I run it every-time I shut down my computer or in between browser closings to eliminate cache files - if there is no cache there is no tracking or profiling since both rely on it

 

[Aristarchus]

If the only parameter modified is vm.swappiness then not much will change. There are other VM parameters that must be adjusted if one wants to change kernel swap policy.
However I would consider ZRAM/ZSWAP as it seems that OP system does not have enough RAM. In such case VM modification is not enough.

Also I doubt that Firefox cookies and history would affect RAM usage mainly because only firefox executable runs in memory (simple to check).
It is easy to force Firefox to use RAM only for cookie policy and disable history. In addition one can modify FF bookmarks backup, recovery from crash and so on but this will only affect disk usage and privacy policy, not RAM usage.

 

[Debian_SuperUser]

@Aristarchus @bob466

I have configured swapiness to a higher value, but only because I have some stability trust issues, so I want the kernel to allocate enough memory to the programs and not conserve too much. And because of that, I would not increase the cache pressure, but maybe I would try a little.

@GatorsFan
It literally doesn't have to do anything with the browser, and I do clean with BleachBit.

Increasing the swapiness is not really the solution of my problem at all. After boot, or after closing the program(s) that were taking lot of memory, I have a lot of free memory, and as they say, "unused RAM is wasted RAM". I am just trying to find a way or a program which will fill up the cache in the background.

 

[AndrewP]

vmtouch is a tool to cache files.
There is the explanation:

https://www.baeldung.com/linux/file-cache-ram

 

[Trenix25]

@osprey

What?? Why does my specs matter at all? I just wanted to know if there is something similar to Sysmain on Linux which can prefetch data as cache.

You can easily cause the system to read in a program by writing your own script to do so that uses the /usr/bin/cat command. You can cat your program and redirect the output to /dev/null. This should cause the system to have read the data and hopefully keep a copy in the cache. How much physical RAM does your computer have? I would recommend only swapping when necessary. I used to use a laptop that only had 3 GB of physical RAM so memory was tight. Now I use a desktop with 16 GB and never run out. As a result I have turned off swapping altogether. I wrote this little program to help me flush the cache whenever I want to.

/*

     usemem.c
     Uses malloc(3) to allocate heap memory and clears it.
     Written by Matthew Campbell.

*/

#include <errno.h>
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

int main( int argc, char **argv )
{
     int save_errno;
     int32_t old_percent, percent;
     int64_t amount, limit, pos, *ptr;

     if ( argc != 2 )
     {
          printf( "Usage: usemem <amount in MiB>\n" );
          return 1;
     }

     if ( sscanf( argv[ 1 ], "%ld", &amount ) != 1 )
     {
          printf( "The amount of memory to use must be an integer number.\n" );
          return 1;
     }

     if ( amount <= 0L )
     {
          printf( "The amount of memory to use must be greater than zero.\n" );
          return 1;
     }

     if ( ( amount * 1024L * 1024L ) > ( 0x7FFFFFFFFFFF / 100L ) )
     {
          printf( "\
Allocating the requested amount may cause a data type overflow.\n" );
          return 1;
     }

     printf( "Attempting to allocate %ld MiB of heap memory...\n", amount );
     limit = amount * 1024L * 1024L;

     if ( limit < amount )
     {
          printf( "Data type overflow.\n" );
          return 1;
     }

     errno = 0;
     ptr = malloc( limit );

     if ( ptr == NULL )
     {
          save_errno = errno;
          printf("Failed to allocate %ld MiB of heap memory.\n", amount );
          if ( save_errno != 0 )
          {
               printf( "Error: %s.\n", strerror( save_errno ) );
          }
          return 1;
     }

     limit = limit / 8L;
     printf( "Heap memory allocated.\n" );
     old_percent = percent = 0;
     printf( "Clearing it to zero... 0%%" );
     fflush( stdout );

     for( pos = 0L; pos < limit; pos++ )
     {
          ptr[ pos ] = ( int64_t )0;
          percent = ( int )( ( pos * 100L ) / limit );

          if ( percent > old_percent )
          {
               old_percent = percent;
               printf( "\rClearing it to zero... %d%%", percent );
               fflush( stdout );
          }

     }    /* for( pos = 0L; pos < limit; pos++ ) */

     printf( "\rClearing it to zero... 100%%\n" );
     printf( "Allocated and cleared %ld MiB of heap memory.\n", amount );
     printf( "Press Enter to release it..." );
     fflush( stdout );
     getchar();

     free( ptr );
     ptr = NULL;

     printf( "Heap memory released.\n" );
     return 0;
}

/* EOF usemem.c */

This is usemem.c. I use it for virtual memory configuration and low memory stress testing. I hope it helps some people.

I would recommend clearing your browser's cache, cookies, and history after each browsing session, for security reasons if nothing else. I once needed to reload a web page on a neighbor's Android phone. I told it to clear the cache and it took ten whole minutes. I timed it. A browser's cache is ripe for data mining if someone wants to see what you've been up to.

You can write a systemd unit file to add a script to run on a systemd timer that will reload all of those files on a regular basis. You could also check to make sure the system load isn't too high first. See /proc/loadavg to find this information. Using cron would work too instead of systemd. I have noticed that when aide runs it fills up the system cache because it reads almost every file on the system. aide uses buffered I/O as evidenced by kswapd0 running along with it. Non-buffered I/O will not involved kswapd0, a kernel thread. You can verify this using /usr/bin/dd with iflag=direct.

Signed,

Matthew Campbell

 

[Trenix25]

@Aristarchus @bob466

I have configured swapiness to a higher value, but only because I have some stability trust issues, so I want the kernel to allocate enough memory to the programs and not conserve too much. And because of that, I would not increase the cache pressure, but maybe I would try a little.

@GatorsFan
It literally doesn't have to do anything with the browser, and I do clean with BleachBit.

Increasing the swapiness is not really the solution of my problem at all. After boot, or after closing the program(s) that were taking lot of memory, I have a lot of free memory, and as they say, "unused RAM is wasted RAM". I am just trying to find a way or a program which will fill up the cache in the background.

You might have a look at /etc/sysctl.conf and in particular the two variables vm.min_free_kbytes and vm.user_reserve_kbytes.

Be careful though. Reserving too little may prevent some programs from loading. 20 MB was too little to reliably load /usr/bin/top.

Signed,

Matthew Campbell

 

[bob466]

Sometimes when I am done with heavy web browsing, the system is not as fast as a fresh reboot. One thing is that a lot of the memory has been paged, but I can just disable and re-enable the swap space to load all the paged memory back in RAM. But the other thing is a lot of my cache is gone, and same thing after boot of course. I have to manually open my programs for them to be cached. Just like Sysmain in Windows, is there a prefetcher for Linux? I tried preload, but that doesn't seem to be doing much.

My Tower has 16GB of Ram and doesn't get your problems...my old Laptop has only 4GB of Ram and doesn't have your problems. Both have a 2GB Swap file and Swappiness is set to 20...both never slow down...both have a 500GB SSD.

You say you use BleachBit for cleaning...this may be your problem because it's like wrecking ball that can damage your system beyond repair...this may have happened.

This is for Mint but is good advice just the same...
https://easylinuxtipsproject.blogspot.com/p/fatal-mistakes.html#ID4

 

[Debian_SuperUser]

@bob466 I don't have any "problems". I am too sensitive to notice that my programs launch just a little late. I should disclose it that I have 8 GB of RAM. Anybody no matter if 16 gigs or 4 gigs of memory does equally run a high memory workload, after closing it they will have all their cache gone. If you do have a faster SSD, then you might not notice it that much.

 

[bob466]

@bob466 I don't have any "problems". I

Then why start this Thread ?

I never had your "problems" when running HDDs either...maybe you think Linux has the same problems windoze does or it's your setup.

after closing it they will have all their cache gone. If you do have a faster SSD, then you might not notice it that much.

This makes no sense to me because from time to time I'll run a Terminal command to clear cache which creates more Disk space...which I don't really need as I have plenty. I'll also clean the thumbnail cache and I clear the updates cache too and my system works just fine.

​

 

[Debian_SuperUser]

Then why start this Thread ?

I don't think it's necessary to only start a thread if you have a real problem. I started this thread to make my experience better, and not particularly fix any problem.

I never had your "problems" when running HDDs either...maybe you think Linux has the same problems windoze does or it's your setup.

Then you are just not sensitive enough or don't care about the performance loss.

This makes no sense to me because from time to time I'll run a Terminal command to clear cache which creates more Disk space...which I don't really need as I have plenty. I'll also clean the thumbnail cache and I clear the updates cache too and my system works just fine.

My guy, that is not the cache I am talking about . Memory caching, not anything with the drive.

 

[MikeWalsh]

@Debian_SuperUser :-

Me, I have 32GB RAM, 5+ TB fast, SSD storage, 64GB swap space - blame HP for that, not me; if you don't suspend/hibernate the entire 32GB of RAM (regardless of the amount used) to a single, contiguous location, it throws a wobbly and refuses to wake up! - with swappiness set to around 10. And that's along with a Pentium quad-core running at almost 4 GHz.

Personally, I just don't concern myself with performance. I doubt a prefetcher would benefit me; using "Puppy" as I do, the system runs like a scalded cat anyway, regardless of what I'm doing with it OR how many apps happen to be open at any given moment in time.....

Even if "free" reports hardly any memory left to use, I know perfectly well that when I want it, Linux will release it.

The system suits me very well.


Mike.

 

[bob466]

My guy, that is not the cache I am talking about .

I don't think you know what you're talking about...that's a shame.

 

[wizardfromoz]

I'll step in here.

Manners people, remember them, please.

Chris Turner
wizardfromoz

 

[bob466]

I'll step in here.

Manners people, remember them, please.

Chris Turner
wizardfromoz