# Notes ## Grymoire Tutorial These are notes from the tutorial on [POSIX shell](https://grymoire.com/unix/Sh.html#toc_Table_of_Contents). There is also a splash from [rwxrob](https://github.com/rwxrob/boost) in here too. I did not cover every topic covered nor are these notes as thorough as the resources used. ## Shell Basics The shell operates in a predictable and linear manner 1. Meta-characters are handled 1. The name of the executable is found 1. Arguments are passed to the program 1. File redirection is setup 1. The program is executed ## File Name Expansion AKA Globbing ### Meta-Characters and Filename Expansion | Pattern | Matches | | ------------- | --------------------------------------------------------------- | | \* | Every file in the current directory | | ? | Files consisting of one character | | ?? | Files consisting of two characters | | ??\* | Files consisting of two or more characters | | \[abcdefg] | Files consisting of a single letter from a to g | | \[a-g] | Same as above | | \[a-cd-g] | Same as above | | \[a-zA-Z0-9] | Files that consist of a single letter or number | | \[!a-zA-Z0-9] | Files that consist of a single character not a letter or number | | \[a-zA-Z]\* | Files that start with a letter | | ?\[a-zA-Z]\* | Files whose second character matches a letter | | \*\[0-9] | Files that end with a number | | ?\[0-9] | Two character filename that end with a number | | \*.\[0-9] | Files that end with a dot and a number | ### File Name Expansion with Directories | Pattern | Matches | | ------- | ------------------------------------------ | | \* | All non-hidden files | | abc/\* | All non-hidden files in the abc directory | | abc/.\* | All hidden files in the abc directory | | \*/\* | All non-hidden files in all subdirectories | | \*/.\* | All hidden files in all subdirectories | Understanding file name expansion is important for security. When you use \* at the command line it will expand the file names and add it to the command. Why would this be important? Let's say we want to run `rm *` and the contents of the director are `0.md 1.md`. We would expect that the command will delete both files without any imput from the user. However, compare that to a directory with `-i 0.md 1.md` and you will have entirely different behavior. The first command expands to `rm 0.md 1.md` while the second expands to `rm -i 0.md 1.md` wand the `-i` is what causes the change in how rm operates. ## Arguments Vs Options The shell does not see either different from each other. It is the program or executable that makes the difference and treats them differently. Let's consider `rm -i 01.md 02.md`. In this example, "-i, 01.md, and 02.md" are all arguments passed to the command rm. By convention programs use a dash "-" to differntiate a program option from a program argument. ## Space as a meta character One meta character that tends to be forgotten about is the space. The shell treats a space as an indicator of the end of a word. Taking the previous example, `rm -i 01.md 02.md` all four components are words. The first word "rm" is the command and the last three are arguments to be passed into that command. ## Quotes ### Using Quotes and the Escape Character Quotes can be used to group text together and can, in the case of the single quote, escape special characters. Assume you want to delete the file with the name: "delete me.md". As discussed above, the space is a meta-charactger that deliniates words. If you were to try to delete it by using `rm delete me.md` the shell will first look for the file "delete" and error out. Instead you need to "escape" the space with either the single quote (''), double quote (""), or the backslash (\\). Here are some examples that will work: rm "file1 file2" rm file1 file2 rm file1" "file2 rm 'file1 file2' rm file1' 'file2 rm f'ile1 file'2 ### Nested Quotes and Escape Character It can become very complicated when trying to nest quotes within quotes. Nesting double quotes ```bash echo '"' echo "\"" ``` Nesting single quotes ```bash echo "'" ``` Nested Escape Character ```bash echo '\' echo "\\" ``` ### Strong vs Weak Quoting The Single quote is seen as a strong quote where the double quote is seen as a weak quote. Strong quoting prevents characters from having any special meaning while weak quoting allows meta-characters to maintain their special meanings. Example: `echo '$HOME'` will display `$HOME` where `echo "$HOME"` will display the home path of the current user. ## Command Substitution (Backquotes) The backquotes' special use is for command substitution. Anything between backquotes is executed by the shell and the results replace the backquoted portion of the command. For example if you want to echo the current working directory you can use: ```bash echo `pwd` ``` This will repalce the pwd with the current working directory the user is in. Another format of command substitution is using `$(pwd)` ## Variables There is a simple syntax for variables =. Notice that there is no space between the variable name and the value of the variable. The space character will terminate the variable. You can set more than one variable on a single line. Exmaples of setting a variable: ```bash A=1 A=1 B=2 C=3 a="This is a value" ``` You can view the values of all variables with the `set` command. ## Inherited Variables To get a variable's value accessible to a subshell, you must `export` that variable. This gives the subshell a variable of the same name and value which can be changed in that subshell but the subshell cannot change the value of the parent shell. ```bash var="My Variable" export var ``` ### IFS Variable This variable lists the characters used to terminate a word. Attackers can modify the IFS variable to gain privileged access to a system. If a program runs `/bin/ps` without using quotes, it is possible that an attacker changes the IFS to include "/" which will have the program run `bin ps`. Now the attacker needs only to put a program called `bin` in the PATH and will gain privileged access. ### PS1 Variable The `PS1` variable specifies the prompt you see on the terminal before each command entered. It is common to see the "$" for non-root users and "#" for root users. ### PS2 Variable The `PS2` variable specifies the prompt you see on secondary prompts. For example, a multiline command. ## Substitution Operations These are good for when you want to do something based on if a variable has a value or not. | Operator | Description | Example | | ------------------- | -------------------------------------------------------------------------------- | --------------------------------- | | ${varName:-value} | Shows the value if a variable is not defined. | $COST:-'not defined' | | ${varName:=value} | Assigns the value to the variable if the variable does not exist. | ${NAME:='Arron'} | | ${varName:value} | Assigns a value if the parameter has no value or it does not exist. | ${LNAME:'PATTON'} | | ${varName:?value} | Shows the error message of the value if the variable does not exist or is empty. | ${NAME:?'is not defined'} | | ${varName:+value} | Shows the message that is defined if the variable has a value. | ${NAME:+"has the value of $NAME"} | | ${varName%value} | Remove the value from the end of the variable. | ${NAME%'Arron'} | | ${varName%%\*value} | Remove the value from the end of the variable (greedy). | ${NAME%%'Arron'} | | ${varName#value} | Remove the value from the beginning of the variable. | ${NAME#'Arron'} | | ${varName##\*value} | Remove the value from the beginning of the variable (greedy). | ${NAME##'Arron'} | ## Positional Parameters $1...$9 Shell scripts can recieve arguments and those arguments are stored in special variables based on the argument position. For example `script arg1 arg2`. The variable $1 would be arg1 and $2 would be arg2. This continues up to and including $9. However, it is possible to pass more than 9 arguments to your script. It will just take a little more work to access those arguments. Some shells such as bash will let you access them using `${10}` POSIX does not. ### Shift The `shift` command allows you to shift the arguments down. We will use this as an example `script arg1 arg2 arg3 arg4 arg5`. ```bash #!/bin/sh a1=$1; shift a2=$1; shift a3=$1; shift a4=$1; shift a5=$1; shift echo "$a1 $a2 $a3 $a4 $a5" ``` ```bash #!/bin/sh a1=$1; a2=$2; a3=$3; a4=$4; a5=$5; shift 5 echo "$a1 $a2 $a3 $a4 $a5" ``` ## Other Special Variables ### $0 This variable contains the script name. This variable is not affected by the shift command. ## $\* This variable contains all of the arguments. When attempting to loop through `"$*"`, it will just return all of the arguments in the first iteration of the loop. ## $@ This variable is much like the `$*` in that it contains all of the arguments; however, `$@` retains spaces in arguments where `$*` does not. When looping through `"$@"`, you will get each individual parameter as it was entered. ## $\# This variable is equal to the number of arguments passed to the script. ## $$ This variable gets the current process id. ## Debugging If you are having issues understanding how a script is functioning you can use the `x` flag of sh to get more information about how the script runs. You can set this in a few different ways. First, you can set it when executing the script `/bin/sh -x script arg1`. Second, you can add it to the shebang `#!/bin/sh -x`. Third, you can turn it on and off as you need to in the script. This is helpful when the script is long and you are only debugging a specific section. To turn it on use `set -x` and to turn it off again use `set +x`. ## Check for Syntax Errors You can use the `-n` argument to have the shell read the script but not execute anything. ## Shell exit Flag By default a script will exit on errors because the shell exit flag is set. To ignore errors you can set `set +e` and to turn it back on again `set -e`. ## Flow Control ### Simplest Form of Flow Control using &&, ||, and ; The simplest way to use flow control is to use && and ||. `command1 && command2` Both commands will run if and only if the first is successful. This can be read "command1 and command2" `command1 || command2` The second command will only run if the first command is NOT successful. This can be read "command1 or command2" ```bash true && echo "This line IS printed." false || echo "This line IS printed." true || echo "This line is NOT printed." false && echo "This line is NOT printed." ``` To control what happens in succession you can use the `;`. `command1; command2; command3`. This will execute command1 then command2 then command3. ### Precedence There is a precedence in operations when it comes to using &, &&, |, ||, and ;. && and || have a higher precendence than ; and & but lower than |. `a|b && c; d||e|f;` will be evaluated to `(a|b(b&&c)); ((d||e)|f)` ### Changing Precedence You can use "()" or "{}" to change the precedence. However, be careful when choosing which to use as there are some major differences. Using "()" causes the shell to execute a new process while "{}" does not. ```bash a=OLD (a=NEW); echo $a { a=NEW; }; echo $a ``` Note the space after the "{" and before the "}" as well as the ";" inside and after the "{}". ## Ways Commands Can be Grouped Simple, pipeline, and lists are the three ways commands can be grouped together. A simple command is a collection of words separated by spaces. A pipeline is a group of simple commands separated by a "|". A list is a series of pipelines, sepatrated by &, ;, &&, or || and terminated by a ;, &, or a new line character. ## If, While, Until ### If The structure of an if statement is: `if list then list fi`. This means after each "list" a newline or semicolon is required. ```bash if true then echo "This was true" fi ``` or ```bash if true; then echo "This was true" fi ``` or ```bash if true; then echo "This was true"; fi ``` This is also true if using elif and else ```bash if true then echo "This is true" elif false then echo "this is flase" else echo "this should never happen" fi ``` ```bash if true; then echo "This is true"; elif false; then echo "This is false"; else echo "This should never happen"; fi ``` ### While Basic struture of while is `while list do list done` ```bash while true do echo "this will run for ever" done ``` ```bash while true; do echo "this will run for ever"; done ``` ### Until Basic structure of until is `until list do list done` ```bash until testa do echo "Not yet" done ``` ```bash until testa; do echo "Not yet"; done ``` ## For and Case ### For For iterates through one or more words executing a list. `for name do list done` `for name in word do list done` ```bash for i in 1 2 3 4 5 6 7 8 9 do echo $i done ``` ```bash for i in 1 2 3 4 5 6 7 8 9; do echo $i; done ``` This example will iterate through both variables of multiple words. It will first iterate through num1 then num2. ```bash num1='1 2 3 4 5' num2=`5 4 3 2 1` for i in num1 num2 do echo $i done ``` You can use command expansion to create the list to iterate through ```bash for i in $(ls ) do echo $i done ``` You can combine variables, constants, and command expansions together to make them one iteration. ```bash for "$a $b $c" do echo $i done ``` ### Case Case functions like a complex if statement with multiple clauses. `case word in pattern | pattern )list;; esac` ```bash echo answer yes or no read word case $word in yes | YES ) echo "You answered yes" ;; no | NO ) echo "You answered no" ;; esac ``` You can use pattern matching as well ```bash echo answer yes or no read word case $word in [Yy]* ) echo "You answered yes" ;; [Nn]* ) echo "You answered no" ;; esac ``` There can only be one word after case. So if you are ever checing more than one word, you need to use a variable. ```bash arg='a b c' case $arg in [aAbBcC] ) echo "This never matches ;; "a b c" ) echo "This will match" ;; esac ``` ## Break and Continue If you want to jump out of a for or while loop you can use the `break` command and if you want to end the current iteration at a specific point you can use the `continue`. ```bash arg='1 2 3 4 5 6 7 8 9' for i in $arg do case "$arg" in 2|4|6|8 ) continue ;; 0 ) break ;; esac echo $arg ``` ## Functions To create a function use the format of `funcName() { list }` This will attempt to increment the argument given to it by one ```bash inc() { echo $(echo $1 + 1 | bc) } echo $A ``` ## Parsing Arguments Using flags that have arguments ```bash #!/bin/sh usage () { echo "$0 Error: $*" 1>&2 echo "usage: $0 [-i IP] [-p PORT]" 1>&2 exit 1 } i='' p='' while : do case "$1" in -i ) shift; i=$1 ;; -p ) shift; p=$1 ;; -* ) usage "Bad argument $1";; * ) break ;; esac shift done echo "You entered the IP: $i and PORT: $p" ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://book.dragonsploit.com/linux/posix/grymoire-tutorial.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.