Why are there no easy ways in Linux to retrieve system information?



Beaver763987450 said:
Thanks all. :)
I think I have some hints on where I should start digging now.


Regarding this, I was hoping there was a way for user-space programs to avoid this parsing issue.
It could make monitoring programs (like htop) work more efficiently and maybe poll the CPU data more frequently without causing performance issues.
Although, it is nice that kernel-space has no such limitations.
And perhaps the performance penalty of this parsing is not too bad.

But if adding more virtual files to Linux is not a problem, perhaps having differently-formatted virtual files like these would be of some value:
Code: 
/proc/stat
/proc/stat.json
/proc/stat.bin

Even as an end-user, I would prefer to read JSON data than to count columns and cross-reference against manuals to figure out what each value means. But it might be because I am lazy. :D
But this will probably require more code in the Linux code-base, and more complexity, so it is not without a cost.

Anyway, I think I will go the htop way for CPU monitoring now. :)
Click to expand...
Anyone can write an virtual filesystem. There's even a File-system-In-User-Space (FUSE) module for writing a filesystem without having to work inside the kernel.

Actually getting something accepted into kernel official source would be pretty hard work. You'd need to sell it right, write it right, and probably establish some creds before even suggesting it. But providing you write your filesystem as a kernel module or a FUSE module, there's no need for it to be officially part of the kernel, it could be distributed separately.

You could also write a D-Bus service that exposes existing kernel data in custom D-Bus service API. This localnet client-server approach moves totally away from files. This is probably the way to go if you're wanting to serve data conveniently to different languages and desktop-environments. If your run the busctrl command in a KDE or Gnome desktop without any arguments, you can see that D-Bus is used quite a lot. So modern Linux has, in some cases, grown away from the old UNIX approach.

To some degree, using files and files containing text probably harks back to the COBOL erra. Data was quite often kept unpacked in BCD (binary coded decimal), quite readable, and machines had BCD instructions. Even typical COBOL processing is similar to UNIX pipes: read a file selecting data, as you do so pass it to a sort to get it into the right order//hierarchy for output, read it out of the sort and format and write it out to an output-file or report - it's kind of resembles grep blah | sort | awk
 
PsychoHermit said:
You can gather all kinds of information if you look in /proc

--glenn
Click to expand...
I think we've come full circle. The OP was asking why it was necessary to trawl a file hierarchy and parse text files.

If one is already coding in C/C++ it can seem a waste of resources to have to trawl this stuff and turn it back into integers, lists, trees, ...
 
Insomniac said:
Marvel at my latest creation of a Linux system monitor and spend hours and days learning how it works. insert evil laughter here
Click to expand...

Been there, done that, it's a right of passage :cool:
Screenshot_20240223_191228.png

Python psutil along with pyqt makes this kind of stuff straight forward enough.

With this utility all processes are shown. Each process is a colored dot indicating the owning user. That's unless a dot is temporarily bright red, which means that process is consuming 100% of a CPU - which results in a Freedesktop-Notification. Or unless a dot is deep-blue which means it's changed the amount of memory it consumes, small dots have recently reduced their memory size, larger dots have recently grown. Dotted circles represent RSS size. An adjacent small green dot indicates that process recently did a read, an adjacent small red dot indicates in recently did a write. Info on some dots, such as the grey ones, is lacking because they belong to a different user, for example the grey dots are root's processes, the blue dots are my own processes. As simple as that - evil laughter here.

I originally started this monitor because chrome had a bug that would silently kick in and consume an entire CPU, but I might not notice for hours. So I wanted something to raise a Freedesktop-Notification if a process persisted with high CPU consumption. Plus I raised alerts for continuous RSS size growth to give me a small window to head of the OOM-killer.

I still use it to make me aware of CPU and memory hogs - but it mainly stays buried in the the system-tray (of KDE). Even though it's python, as long as it's minimised to the tray, it has minimal overheads - it proves trawling /proc using python psutil isn't too expensive (at least not on modern CPU's).
 
My reason for making my system monitor they way I did was that I found myself often opening terminals for precisely this information. Now I no longer have to.

Colored dots are cute, but not the precise information I need to make decisions while inserting/removing hardware, making backups etc. Dots can't replace this amount of useful information.

Classic Sysadmin vs programmer :p
 
Insomniac said:
My reason for making my system monitor they way I did was that I found myself often opening terminals for precisely this information. Now I no longer have to.

Colored dots are cute, but not the precise information I need to make decisions while inserting/removing hardware, making backups etc. Dots can't replace this amount of useful information.

Classic Sysadmin vs programmer :p
Click to expand...
Same underlying data, but massaged differently for different purposes - as it should be.
 
digitaltrails said:
Same underlying data, but massaged differently for different purposes - as it should be.
Click to expand...
Semantics and excuses for failing to address or even recognize the users needs, as most programmers do.

I've had the pleasure of working with plenty of talented people in my life including programmers. They're not all bad. But a lot seem oblivious to what a real world looks like, what people do in it, and what their needs are.

Trying to communicate with programmers is something I wouldn't wish on my worst enemy. It's worse than herding cats because you know they're capable of intelligence.
 
Last edited by a moderator:
Quick reminder...

Our rules can be distilled to 'civility'. Even when disagreeing, we must be civil.
 
wizardfromoz said:
@JasKinasis might have some input in this regard, as well. By mentioning him, I have brought it to his attention.

Cheers

Wizard
Click to expand...
Late to the party.

I have to admit, I haven’t created any low level system tools on Linux. At least not in C, or C++. Most of my C/C++ programs for Linux have just been ordinary terminal, or desktop applications for my own use.

At a previous job, I also ported some in-house, windows only, MFC applications to QT or wxWidgets, in order to make them cross platform. Some of which were originally written in Visual Basic, so I had to completely re-write them in C++. But they were all desktop applications. Not a lot of low level system stuff in there.

For the last 16 years, I’ve been working for a company who produce Windows only CAD/CAM software for the shoemaking industry. So again, desktop applications, but there is occasionally some low level stuff, but it’s usually dealing with interfacing with certain types of custom hardware.

For system monitoring purposes in Linux, I usually just use Bash one liners, aliases, or shell-scripts for viewing system information.

So I use whatever terminal based tools are available for obtaining the information I’m interested in and then filter/process their output using the usual tools (ag/grep, sed, awk, cut etc).

I’m not aware of any existing system monitoring tools that output structured data in parsable formats like .json, or xml, or machine readable binary formats. Usually it’s just plain text. The reason I use shell-scripts is because there are tons of useful, pre-existing tools for parsing and processing text in the terminal. It saves me from having to manually parse and process textual information in a C/C++ program.

You could say it’s a pragmatic choice. Or you could just say I’m really lazy!!
 
Last edited:
JasKinasis said:
I’m not aware of any existing system monitoring tools that output structured data in parsable formats like .json, or xml, or machine readable binary formats. Usually it’s just plain text.
Click to expand...

Some stuff, depending on our definitions, has JSON output - like lsblk.

(I read a lot of man pages.)
 
KGIII said:
Some stuff, depending on our definitions, has JSON output - like lsblk.

(I read a lot of man pages.)
Click to expand...
Well, I didn’t say there weren’t any. Only that I was unaware of any. But that’s good to know. Thanks!
 
JasKinasis said:
Well, I didn’t say there weren’t any. Only that I was unaware of any. But that’s good to know. Thanks!
Click to expand...

Oh, no... No, I'm just trying to be helpful.

It looks like some of the ls-type commands (excluding the ls command) do this. I didn't check them all, but JSON is an option with lshw and lscpu both do this. lscpu has nothing of the sort.
 
@Beaver763987450

You have been offered a bunch of tools, since all these tools are open source you might as well look into them and borrow code which you need in your program.

Yes there are no default system API's, I suggest you approach programming from 2 points here, write a library of API's that harvest system info, and write separately your program which will use it.
 
KGIII said:
Quick reminder...

Our rules can be distilled to 'civility'. Even when disagreeing, we must be civil.
Click to expand...
Show me on the civility doll where I touched anyone. Take your time, it must be difficult, I feel for your loss.

In this exchange we both expressed our ideas, disagreed, and moved on with our lives. This sort of exchange happens daily everywhere.

So where is this perceived problem exactly?
 
Last edited by a moderator:
Insomniac said:
Show me on the civility doll where I touched anyone. Take your time, it must be difficult, I feel for your loss.

In this exchange we both expressed our ideas, disagreed, and moved on with our lives. This sort of exchange happens daily everywhere.

So where is this perceived problem exactly?
Click to expand...

Specifically?

"Trying to communicate with programmers is something I wouldn't wish on my worst enemy. It's worse than herding cats because you know they're capable of intelligence."

But you knew that already.

Like you claim to be, I'm capable of just moving along. In fact, I'd long since forgotten this. It's really too late in the day to make this an issue unless you really feel inclined to do so. Or, you know, you can just be civil.

That and we can all use a friendly reminder to remain civil in public discourse. Especially when the goal is a bit of technical improvement for ourselves and others.
 
  • Like
Reactions: Rob
What does the account deletion procedure look like for Linux.org and why is it not in the main profile?
 
Insomniac said:
Show me on the civility doll where I touched anyone. Take your time, it must be difficult, I feel for your loss.

In this exchange we both expressed our ideas, disagreed, and moved on with our lives. This sort of exchange happens daily everywhere.

So where is this perceived problem exactly?
Click to expand...

Before posting catty remarks you might want to consider that you're getting free help from professionals.

Outside of this forum, a lot of these people get paid to answer questions like yours. If I were a professional offering free help to someone who doesn't appreciate my time and expertise that person be pretty low on my list of priorities.
 
You must log in or register to reply here.

Members online

Total: 468 (members: 3, guests: 465)

Latest posts

Top