Useful Linux Commands
1
# add text to the start of a line
2
sed -i -e 's_.*_REPLACETHIS&_' <filename>
Copied!
1
# Will remove the word Javascript from the file cewlfile
2
cat cewlfile | tr -d \[:CHAR:Javascript\]
Copied!

RTFM

Useful tool help remind you of command syntax

1
git clone https://github.com/leostat/rtfm
2
cd rtfm
3
pip3 install terminaltables
4
./rtfm.py -e hydra #EXAMPLE
Copied!
1
# 1. redo last command but as root
2
sudo !!
3
4
# 2. open an editor to run a command
5
ctrl+x+e
6
7
# 3. create a super fast ram disk
8
mkdir -p /mnt/ram
9
mount -t tmpfs tmpfs /mnt/ram -o size=8192M
10
11
# 4. don't add command to history (note the leading space)
12
ls -l
13
14
# 5. fix a really long command that you messed up
15
fc
16
17
# 6. tunnel with ssh (local port 3337 -> remote host's 127.0.0.1 on port 6379)
18
ssh -L 3337:127.0.0.1:6379 [email protected] -N
19
20
# 7. quickly create folders
21
mkdir -p folder/{sub1,sub2}/{sub1,sub2,sub3}
22
23
# 8. intercept stdout and log to file
24
cat file | tee -a log | cat > /dev/null
25
26
# bonus: exit terminal but leave all processes running
27
disown -a && exit
Copied!

Commands Explained

1
https://explainshell.com/
Copied!

Kill a process using a port

1
lsof -t -i:443
2
kill -9 $PID
Copied!

Append to end of file

1
echo 'text here' >> filename
Copied!

sed

1
FILE SPACING:
2
# remove white spacing from a file
3
sed "s/^[ \t]*//" -i $filename
4
5
# extract ports from nmap scan
6
sed 's/\/tcp.*//g' ports
7
# then use this command to pring out ports as comma seperated values
8
awk -vORS=, '{ print $1 }' ports | sed 's/,$/\n/'
9
10
# double space a file
11
sed G
12
13
# double space a file which already has blank lines in it. Output file
14
# should contain no more than one blank line between lines of text.
15
sed '/^$/d;G'
16
17
# triple space a file
18
sed 'G;G'
19
20
# undo double-spacing (assumes even-numbered lines are always blank)
21
sed 'n;d'
22
23
# insert a blank line above every line which matches "regex"
24
sed '/regex/{x;p;x;}'
25
26
# insert a blank line below every line which matches "regex"
27
sed '/regex/G'
28
29
# insert a blank line above and below every line which matches "regex"
30
sed '/regex/{x;p;x;G;}'
31
32
NUMBERING:
33
34
# number each line of a file (simple left alignment). Using a tab (see
35
# note on '\t' at end of file) instead of space will preserve margins.
36
sed = filename | sed 'N;s/\n/\t/'
37
38
# number each line of a file (number on left, right-aligned)
39
sed = filename | sed 'N; s/^/ /; s/ *\(.\{6,\}\)\n/\1 /'
40
41
# number each line of file, but only print numbers if line is not blank
42
sed '/./=' filename | sed '/./N; s/\n/ /'
43
44
# count lines (emulates "wc -l")
45
sed -n '$='
46
47
TEXT CONVERSION AND SUBSTITUTION:
48
49
# IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format.
50
sed 's/.$//' # assumes that all lines end with CR/LF
51
sed 's/^M$//' # in bash/tcsh, press Ctrl-V then Ctrl-M
52
sed 's/\x0D$//' # works on ssed, gsed 3.02.80 or higher
53
54
# IN UNIX ENVIRONMENT: convert Unix newlines (LF) to DOS format.
55
sed "s/$/`echo -e \\\r`/" # command line under ksh
56
sed 's/#x27;"/`echo \\\r`/" # command line under bash
57
sed "s/$/`echo \\\r`/" # command line under zsh
58
sed 's/$/\r/' # gsed 3.02.80 or higher
59
60
# IN DOS ENVIRONMENT: convert Unix newlines (LF) to DOS format.
61
sed "s/$//" # method 1
62
sed -n p # method 2
63
64
# IN DOS ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format.
65
# Can only be done with UnxUtils sed, version 4.0.7 or higher. The
66
# UnxUtils version can be identified by the custom "--text" switch
67
# which appears when you use the "--help" switch. Otherwise, changing
68
# DOS newlines to Unix newlines cannot be done with sed in a DOS
69
# environment. Use "tr" instead.
70
sed "s/\r//" infile >outfile # UnxUtils sed v4.0.7 or higher
71
tr -d \r <infile >outfile # GNU tr version 1.22 or higher
72
73
# delete leading whitespace (spaces, tabs) from front of each line
74
# aligns all text flush left
75
sed 's/^[ \t]*//' # see note on '\t' at end of file
76
77
# delete trailing whitespace (spaces, tabs) from end of each line
78
sed 's/[ \t]*$//' # see note on '\t' at end of file
79
80
# delete BOTH leading and trailing whitespace from each line
81
sed 's/^[ \t]*//;s/[ \t]*$//'
82
83
# insert 5 blank spaces at beginning of each line (make page offset)
84
sed 's/^/ /'
85
86
# align all text flush right on a 79-column width
87
sed -e :a -e 's/^.\{1,78\}$/ &/;ta' # set at 78 plus 1 space
88
89
# center all text in the middle of 79-column width. In method 1,
90
# spaces at the beginning of the line are significant, and trailing
91
# spaces are appended at the end of the line. In method 2, spaces at
92
# the beginning of the line are discarded in centering the line, and
93
# no trailing spaces appear at the end of lines.
94
sed -e :a -e 's/^.\{1,77\}$/ & /;ta' # method 1
95
sed -e :a -e 's/^.\{1,77\}$/ &/;ta' -e 's/\( *\)\1/\1/' # method 2
96
97
# substitute (find and replace) "foo" with "bar" on each line
98
sed 's/foo/bar/' # replaces only 1st instance in a line
99
sed 's/foo/bar/4' # replaces only 4th instance in a line
100
sed 's/foo/bar/g' # replaces ALL instances in a line
101
sed 's/\(.*\)foo\(.*foo\)/\1bar\2/' # replace the next-to-last case
102
sed 's/\(.*\)foo/\1bar/' # replace only the last case
103
104
# substitute "foo" with "bar" ONLY for lines which contain "baz"
105
sed '/baz/s/foo/bar/g'
106
107
# substitute "foo" with "bar" EXCEPT for lines which contain "baz"
108
sed '/baz/!s/foo/bar/g'
109
110
# change "scarlet" or "ruby" or "puce" to "red"
111
sed 's/scarlet/red/g;s/ruby/red/g;s/puce/red/g' # most seds
112
gsed 's/scarlet\|ruby\|puce/red/g' # GNU sed only
113
114
# reverse order of lines (emulates "tac")
115
# bug/feature in HHsed v1.5 causes blank lines to be deleted
116
sed '1!G;h;$!d' # method 1
117
sed -n '1!G;h;$p' # method 2
118
119
# reverse each character on the line (emulates "rev")
120
sed '/\n/!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//'
121
122
# join pairs of lines side-by-side (like "paste")
123
sed '$!N;s/\n/ /'
124
125
# if a line ends with a backslash, append the next line to it
126
sed -e :a -e '/\\$/N; s/\\\n//; ta'
127
128
# if a line begins with an equal sign, append it to the previous line
129
# and replace the "=" with a single space
130
sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D'
131
132
# add commas to numeric strings, changing "1234567" to "1,234,567"
133
gsed ':a;s/\B[0-9]\{3\}\>/,&/;ta' # GNU sed
134
sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta' # other seds
135
136
# add commas to numbers with decimal points and minus signs (GNU sed)
137
gsed -r ':a;s/(^|[^0-9.])([0-9]+)([0-9]{3})/\1\2,\3/g;ta'
138
139
# add a blank line every 5 lines (after lines 5, 10, 15, 20, etc.)
140
gsed '0~5G' # GNU sed only
141
sed 'n;n;n;n;G;' # other seds
142
143
SELECTIVE PRINTING OF CERTAIN LINES:
144
145
# print first 10 lines of file (emulates behavior of "head")
146
sed 10q
147
148
# print first line of file (emulates "head -1")
149
sed q
150
151
# print the last 10 lines of a file (emulates "tail")
152
sed -e :a -e '$q;N;11,$D;ba'
153
154
# print the last 2 lines of a file (emulates "tail -2")
155
sed '$!N;$!D'
156
157
# print the last line of a file (emulates "tail -1")
158
sed '$!d' # method 1
159
sed -n '$p' # method 2
160
161
# print the next-to-the-last line of a file
162
sed -e '$!{h;d;}' -e x # for 1-line files, print blank line
163
sed -e '1{$q;}' -e '$!{h;d;}' -e x # for 1-line files, print the line
164
sed -e '1{$d;}' -e '$!{h;d;}' -e x # for 1-line files, print nothing
165
166
# print only lines which match regular expression (emulates "grep")
167
sed -n '/regexp/p' # method 1
168
sed '/regexp/!d' # method 2
169
170
# print only lines which do NOT match regexp (emulates "grep -v")
171
sed -n '/regexp/!p' # method 1, corresponds to above
172
sed '/regexp/d' # method 2, simpler syntax
173
174
# print the line immediately before a regexp, but not the line
175
# containing the regexp
176
sed -n '/regexp/{g;1!p;};h'
177
178
# print the line immediately after a regexp, but not the line
179
# containing the regexp
180
sed -n '/regexp/{n;p;}'
181
182
# print 1 line of context before and after regexp, with line number
183
# indicating where the regexp occurred (similar to "grep -A1 -B1")
184
sed -n -e '/regexp/{=;x;1!p;g;$!N;p;D;}' -e h
185
186
# grep for AAA and BBB and CCC (in any order)
187
sed '/AAA/!d; /BBB/!d; /CCC/!d'
188
189
# grep for AAA and BBB and CCC (in that order)
190
sed '/AAA.*BBB.*CCC/!d'
191
192
# grep for AAA or BBB or CCC (emulates "egrep")
193
sed -e '/AAA/b' -e '/BBB/b' -e '/CCC/b' -e d # most seds
194
gsed '/AAA\|BBB\|CCC/!d' # GNU sed only
195
196
# print paragraph if it contains AAA (blank lines separate paragraphs)
197
# HHsed v1.5 must insert a 'G;' after 'x;' in the next 3 scripts below
198
sed -e '/./{H;$!d;}' -e 'x;/AAA/!d;'
199
200
# print paragraph if it contains AAA and BBB and CCC (in any order)
201
sed -e '/./{H;$!d;}' -e 'x;/AAA/!d;/BBB/!d;/CCC/!d'
202
203
# print paragraph if it contains AAA or BBB or CCC
204
sed -e '/./{H;$!d;}' -e 'x;/AAA/b' -e '/BBB/b' -e '/CCC/b' -e d
205
gsed '/./{H;$!d;};x;/AAA\|BBB\|CCC/b;d' # GNU sed only
206
207
# print only lines of 65 characters or longer
208
sed -n '/^.\{65\}/p'
209
210
# print only lines of less than 65 characters
211
sed -n '/^.\{65\}/!p' # method 1, corresponds to above
212
sed '/^.\{65\}/d' # method 2, simpler syntax
213
214
# print section of file from regular expression to end of file
215
sed -n '/regexp/,$p'
216
217
# print section of file based on line numbers (lines 8-12, inclusive)
218
sed -n '8,12p' # method 1
219
sed '8,12!d' # method 2
220
221
# print line number 52
222
sed -n '52p' # method 1
223
sed '52!d' # method 2
224
sed '52q;d' # method 3, efficient on large files
225
226
# beginning at line 3, print every 7th line
227
gsed -n '3~7p' # GNU sed only
228
sed -n '3,${p;n;n;n;n;n;n;}' # other seds
229
230
# print section of file between two regular expressions (inclusive)
231
sed -n '/Iowa/,/Montana/p' # case sensitive
232
233
SELECTIVE DELETION OF CERTAIN LINES:
234
235
# print all of file EXCEPT section between 2 regular expressions
236
sed '/Iowa/,/Montana/d'
237
238
# delete duplicate, consecutive lines from a file (emulates "uniq").
239
# First line in a set of duplicate lines is kept, rest are deleted.
240
sed '$!N; /^\(.*\)\n\1$/!P; D'
241
242
# delete duplicate, nonconsecutive lines from a file. Beware not to
243
# overflow the buffer size of the hold space, or else use GNU sed.
244
sed -n 'G; s/\n/&&/; /^\([ -~]*\n\).*\n\1/d; s/\n//; h; P'
245
246
# delete all lines except duplicate lines (emulates "uniq -d").
247
sed '$!N; s/^\(.*\)\n\1$/\1/; t; D'
248
249
# delete the first 10 lines of a file
250
sed '1,10d'
251
252
# delete the last line of a file
253
sed '$d'
254
255
# delete the last 2 lines of a file
256
sed 'N;$!P;$!D;$d'
257
258
# delete the last 10 lines of a file
259
sed -e :a -e '$d;N;2,10ba' -e 'P;D' # method 1
260
sed -n -e :a -e '1,10!{P;N;D;};N;ba' # method 2
261
262
# delete every 8th line
263
gsed '0~8d' # GNU sed only
264
sed 'n;n;n;n;n;n;n;d;' # other seds
265
266
# delete lines matching pattern
267
sed '/pattern/d'
268
269
# delete ALL blank lines from a file (same as "grep '.' ")
270
sed '/^$/d' # method 1
271
sed '/./!d' # method 2
272
273
# delete all CONSECUTIVE blank lines from file except the first; also
274
# deletes all blank lines from top and end of file (emulates "cat -s")
275
sed '/./,/^$/!d' # method 1, allows 0 blanks at top, 1 at EOF
276
sed '/^$/N;/\n$/D' # method 2, allows 1 blank at top, 0 at EOF
277
278
# delete all CONSECUTIVE blank lines from file except the first 2:
279
sed '/^$/N;/\n$/N;//D'
280
281
# delete all leading blank lines at top of file
282
sed '/./,$!d'
283
284
# delete all trailing blank lines at end of file
285
sed -e :a -e '/^\n*$/{$d;N;ba' -e '}' # works on all seds
286
sed -e :a -e '/^\n*$/N;/\n$/ba' # ditto, except for gsed 3.02.*
287
288
# delete the last line of each paragraph
289
sed -n '/^$/{p;h;};/./{x;/./p;}'
290
291
SPECIAL APPLICATIONS:
292
293
# remove nroff overstrikes (char, backspace) from man pages. The 'echo'
294
# command may need an -e switch if you use Unix System V or bash shell.
295
sed "s/.`echo \\\b`//g" # double quotes required for Unix environment
296
sed 's/.^H//g' # in bash/tcsh, press Ctrl-V and then Ctrl-H
297
sed 's/.\x08//g' # hex expression for sed 1.5, GNU sed, ssed
298
299
# get Usenet/e-mail message header
300
sed '/^$/q' # deletes everything after first blank line
301
302
# get Usenet/e-mail message body
303
sed '1,/^$/d' # deletes everything up to first blank line
304
305
# get Subject header, but remove initial "Subject: " portion
306
sed '/^Subject: */!d; s///;q'
307
308
# get return address header
309
sed '/^Reply-To:/q; /^From:/h; /./d;g;q'
310
311
# parse out the address proper. Pulls out the e-mail address by itself
312
# from the 1-line return address header (see preceding script)
313
sed 's/ *(.*)//; s/>.*//; s/.*[:<] *//'
314
315
# add a leading angle bracket and space to each line (quote a message)
316
sed 's/^/> /'
317
318
# delete leading angle bracket & space from each line (unquote a message)
319
sed 's/^> //'
320
321
# remove most HTML tags (accommodates multiple-line tags)
322
sed -e :a -e 's/<[^>]*>//g;/</N;//ba'
323
324
# extract multi-part uuencoded binaries, removing extraneous header
325
# info, so that only the uuencoded portion remains. Files passed to
326
# sed must be passed in the proper order. Version 1 can be entered
327
# from the command line; version 2 can be made into an executable
328
# Unix shell script. (Modified from a script by Rahul Dhesi.)
329
sed '/^end/,/^begin/d' file1 file2 ... fileX | uudecode # vers. 1
330
sed '/^end/,/^begin/d' "[email protected]" | uudecode # vers. 2
331
332
# sort paragraphs of file alphabetically. Paragraphs are separated by blank
333
# lines. GNU sed uses \v for vertical tab, or any unique char will do.
334
sed '/./{H;d;};x;s/\n/={NL}=/g' file | sort | sed '1s/={NL}=//;s/={NL}=/\n/g'
335
gsed '/./{H;d};x;y/\n/\v/' file | sort | sed '1s/\v//;y/\v/\n/'
336
337
# zip up each .TXT file individually, deleting the source file and
338
# setting the name of each .ZIP file to the basename of the .TXT file
339
# (under DOS: the "dir /b" switch returns bare filenames in all caps).
340
echo @echo off >zipup.bat
341
dir /b *.txt | sed "s/^\(.*\)\.TXT/pkzip -mo \1 \1.TXT/" >>zipup.bat
Copied!

Copy output of command to clipboard

1
xclip -sel c < cat file.txt
Copied!
Then Ctrl v to paste You may have to install xclip
1
apt-get install xclip
Copied!

Crontab

  • Sytanx:
    1
    minute hour day-month month day-week CMD
    Copied!
  • 1 * * * * $ cmd runs every minute

Extracting archives

  • 1
    tar xvfj test.tar.bz2
    2
    tar zxvf test.tar.gz
    3
    tar zxvf test.tar
    4
    gzip -d test.gz
    5
    unzip test.zip
    6
    zcat rockyou.txt.gz > rockyou.txt
    Copied!

Compressing archives

  • 1
    tar -zcvf test.tar test
    2
    gzip test
    3
    zip -9 test.zip test
    4
    zip -r test.zip test/
    Copied!

Copy files remotely

File Permissions

  • t rwx rwx rwx type / owner / group / world
  • Type is directory (d) or file (-).
  • read (r) 4
  • write (w) 2
  • execute (e) 1
  • chmod 755 test.sh Make a file executable
  • find . -name "*.php" -type f -exec chmod 755 {} \;

Finding Files

  • Update the database of file names on the system
    1
    updatedb
    Copied!
  • Reads the database and shows the location of a file
    1
    locate file
    Copied!
  • Show the path where the app is executed from
    1
    which app
    Copied!
  • Show all the files that start with sbd
    1
    find / -name sbd*
    Copied!
  • Show all world readable directories
    1
    find / -perm -o+w -type d
    Copied!
  • Show all world executable directories
    1
    find / -perm -o+e -type d
    Copied!

Searching within files

  • Search for the pattern xxx in a fil
    1
    grep xxx file
    Copied!
  • Search recursively for a pattern in a director
    1
    grep -r
    Copied!
  • Search for a pattern in a zip file
    1
    zgrep
    Copied!

Locate, Which, Find

Locate

1
updatedb # builds local database of all files on system
2
$ locate nc.exe
3
/usr/share/windows-binaries/nc.exe
4
Copied!

Which

Searches directories that are defined in $PATH
1
$ which ssh
2
/usr/bin/ssh
Copied!

Find

More aggressive search tool that can recursively search any given path for various files
1
$ find / -name nc.exe
2
/usr/share/windows-binaries/nc.exe
3
/usr/share/seclists/Web-Shells/FuzzDB/nc.exe
Copied!

Others

  • Import CA in java store
    1
    sudo keytool -import -alias foo -trustcacerts -keystore cacerts -file cacert.der
    Copied!

curl

display webpage to terminal - text
1
curl $ip -s -L | html2text -width '99' | uniq
Copied!

Whats running on a port

1
ss -lptn 'sport = :443'
Copied!