Introduction
So recently, I was given a practical task at work, where I am currently learning Linux. I can't and won't share any exact information on the whole task and how I completed it, but there was one part I wasn't able to solve, and I wanted to hear some guidance from the community on how to handle this type of problem in the future. I don't have any access to the machine after the task has ended, so I won't be able to test any solutions to see if they work. I just need more general guidelines on how to handle such situations. If you can't answer the question but have any reading material regarding this topic, please share. During the test, I was allowed to use any sources of information, including AI, Google, etc., but it still didn't help me. So let me describe the problem:
Problem Description
I'm connected to a machine through SSH, logged in as a user that has sudo privileges but is not root. I am done with all the written tasks in the list, but I also have an extra task along the way: I need to fix all the "bad" stuff happening meanwhile during my work on the machine. The thing that starts happening is that my terminal window gets some of the specific symbols replaced by spaces. I checked processes with $ ps aux, but it was a couple of times until I saw the process appear that had a corresponding user of "service+":
I couldn't figure out if this is something that is okay, and if it's not, how do I "inspect it"? The process ended shortly, and I looked up $ pstree -p, but the process was a child of:
So I am not sure what has to be done further.
I checked active cron jobs, /var/spool/cron, /etc/crontab, and looked for any suspicious stuff in journalctl -xe.
I found these logs, and they do look suspicious. I am not sure if they are actually the problem. The login from the user vagrant, I believe, is the script that is checking my progress and updating the % I achieved, writing it to a file:
This was displayed before my username in the terminal.
As I understand from ChatGPT: systemd-logind manages user logins, including session handling. This indicates that session 656 for a user (likely vagrant) was closed.
So, what would be a good approach to find and check what's actually running and causing problems?
So recently, I was given a practical task at work, where I am currently learning Linux. I can't and won't share any exact information on the whole task and how I completed it, but there was one part I wasn't able to solve, and I wanted to hear some guidance from the community on how to handle this type of problem in the future. I don't have any access to the machine after the task has ended, so I won't be able to test any solutions to see if they work. I just need more general guidelines on how to handle such situations. If you can't answer the question but have any reading material regarding this topic, please share. During the test, I was allowed to use any sources of information, including AI, Google, etc., but it still didn't help me. So let me describe the problem:
Problem Description
I'm connected to a machine through SSH, logged in as a user that has sudo privileges but is not root. I am done with all the written tasks in the list, but I also have an extra task along the way: I need to fix all the "bad" stuff happening meanwhile during my work on the machine. The thing that starts happening is that my terminal window gets some of the specific symbols replaced by spaces. I checked processes with $ ps aux, but it was a couple of times until I saw the process appear that had a corresponding user of "service+":
Bash:
service+ 28831 1.3 2.9 22256 13372 ? Ss 14:34 0:00 /usr/lib/systemd/systemd --user
service+ 28840 0.0 0.7 174584 3480 ? S 14:34 0:00 (sd-pam)
I couldn't figure out if this is something that is okay, and if it's not, how do I "inspect it"? The process ended shortly, and I looked up $ pstree -p, but the process was a child of:
Code:
systemd(1)-systemd(5384)───(sd-pam)(5387)
So I am not sure what has to be done further.
I checked active cron jobs, /var/spool/cron, /etc/crontab, and looked for any suspicious stuff in journalctl -xe.
Less:
Jan 21 14:54:04 LA-7093 systemd-logind[685]: Removed session 656.
░░ Subject: Session 656 has been terminated
░░ Defined-By: systemd
░░ Support: https://support.oracle.com
░░ Documentation: sd-login(3)
░░
░░ A session with the ID 656 has been terminated.
Jan 21 14:54:04 LA-7093 sshd[30594]: Accepted publickey for vagrant from 10.0.2.2 port 45262 ssh2: RSA SHA256:**removed**
Jan 21 14:54:04 LA-7093 systemd-logind[685]: New session 657 of user vagrant.
░░ Subject: A new session 657 has been created for user vagrant
░░ Defined-By: systemd
░░ Support: https://support.oracle.com
░░ Documentation: sd-login(3)
░░
░░ A new session with the ID 657 has been created for the user vagrant.
░░
░░ The leading process of the session is 30594.
Jan 21 14:54:04 LA-7093 systemd[1]: Started Session 657 of User vagrant.
░░ Subject: A start job for unit session-657.scope has finished successfully
░░ Defined-By: systemd
░░ Support: https://support.oracle.com
░░
░░ A start job for unit session-657.scope has finished successfully.
░░
░░ The job identifier is 63402.
Jan 21 14:54:04 LA-7093 sshd[30594]: pam_unix(sshd:session): session opened for user vagrant(uid=1000) by (uid=0)
Jan 21 14:54:04 LA-7093 sudo[30598]: vagrant : PWD=/home/vagrant ; USER=root ; COMMAND=/bin/bash -c echo -n 79 > /var/progress
Jan 21 14:54:04 LA-7093 sudo[30598]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1000)
Jan 21 14:54:04 LA-7093 sudo[30598]: pam_unix(sudo:session): session closed for user root
Jan 21 14:54:04 LA-7093 sshd[30597]: Received disconnect from 10.0.2.2 port 45262:11: disconnected by user
Jan 21 14:54:04 LA-7093 sshd[30597]: Disconnected from user vagrant 10.0.2.2 port 45262
Jan 21 14:54:04 LA-7093 sshd[30594]: pam_unix(sshd:session): session closed for user vagrant
Jan 21 14:54:04 LA-7093 systemd[1]: session-657.scope: Deactivated successfully.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: https://support.oracle.com
░░
░░ The unit session-657.scope has successfully entered the 'dead' state.
Jan 21 14:54:04 LA-7093 systemd-logind[685]: Session 657 logged out. Waiting for processes to exit.
Jan 21 14:54:04 LA-7093 systemd-logind[685]: Removed session 657.
░░ Subject: Session 657 has been terminated
░░ Defined-By: systemd
░░ Support: https://support.oracle.com
░░ Documentation: sd-login(3)
░░
░░ A session with the ID 657 has been terminated.
I found these logs, and they do look suspicious. I am not sure if they are actually the problem. The login from the user vagrant, I believe, is the script that is checking my progress and updating the % I achieved, writing it to a file:
Code:
Jan 21 14:54:04 LA-7093 sudo[30598]: vagrant : PWD=/home/vagrant ; USER=root ; COMMAND=/bin/bash -c echo -n 79 > /var/progress
This was displayed before my username in the terminal.
As I understand from ChatGPT: systemd-logind manages user logins, including session handling. This indicates that session 656 for a user (likely vagrant) was closed.
What Else I Tried
- Checking .bashrc, .bash_profile, aliases, and jobs -l.
- Inspecting process folders in /proc/<pid>/.
- $ clear; reset.
- Monitoring using $ top.
- Running systemctl list-units --type=service | grep running.
- Using lsof /dev/tty.
- Checking the list of users:
Code:
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
... [shortened for brevity]
vagrant:x:1000:1000::/home/vagrant:/bin/bash
**removed**:x:1001:1001::/home/*removed**:/bin/bash - the profile I was doing the current task under
*removed**:x:1002:1002::/home/*removed**:/sbin/nologin - the account I created during one of the tasks as it was asked. Doesn't have any privileges.
So, what would be a good approach to find and check what's actually running and causing problems?