Shell: Misc Commands

In this tutorial I will show a few useful commands when working with linux shell.

Check Directory Exists:

if [ -d /opt/test/ ]; then
    echo 'Directory Exists'
fi

Check Directory Not Exists:

if [ ! -d /opt/test/ ]; then
    echo 'Directory Does Not Exist'
fi

Check File Exists:

if [ -f /opt/test/test.log ]; then
    echo 'File Exists'
fi

Check File Not Exists:

if [ ! -f /opt/test/test.log ]; then
    echo 'File Does Not Exist'
fi

Lowercase Variable:

val='TEXT'
echo "${val,,}"

Echo Variable:
This will print the value of “test”. Notice we use double quotes.

test='My Test Val'
echo "$test"

Echo String:

echo 'My test string'

Split:
This will split on the comma into an array list and then loop through it.

test='test 1,test 2'
split_test=(${test//,/ })

for val in "${split_test[@]}"
do
    echo $val
done

Date:
This will print the date in the format YYYY-MM-dd

my_date="$(date +Y-%m-%d)"
echo "$my_date"

Remove Space From Variable:

VAL='test string'
echo "${VAL//\ /}"

Increment Variable:

index=0
index=$((index+1))

Substring

VAL='test string'
echo "${VAL:4:4}"

If value is equal to

VAL='hello'
if [ "$VAL" == 'hello' ] ; then
    echo 'Is Equal'
fi

If with OR

VAL='hello'
if [ "$VAL" == 'hello' ] || [ "$VAL" != 'hi' ] ; then
    echo 'Is Hello'
fi

If Variable is Empty

VAL=''
if [ -z "$VAL" ] ; then
    echo 'Is Empty'
fi

Append to File

echo 'Hi' >> file_to_log_to.log

Write to File

echo 'Hi' > file_to_log_to.log

While Loop: Increment to 10

This will loop till the value is 9 then exit.

i=0
while [ $i -lt 10 ];
do
    echo "$i"
done

whoami

USER=$(whoami)

If Variable Contains Text

VAL='my Test String'
if [[ "${VAL,,}" == *"test"* ]] ; then
    echo "Found test"
fi

Color Coding

NoColor=$'\033[0m'
READ=$'\033[0;31m'
GREEN=$'\033[0;32m'
YELLOW=$'\033[1;33;40m'

printf "%s Variable Not Set %s\n" "${RED}" "${NoColor}"

Get Log to a logfile and console

SOME_COMMAND 2>&1 | tee -a "${LOG_FILE_PATH}"

Read a JSON config

JSON=$(cat "/path/to/json/file.json")
export MY_VAR=$(echo "${JSON}" | python -c 'import json,sys;obj=json.load(sys.stdin);print(obj["MyKey"])')

Extract tar to Folder

sudo tar -xvf /the/location/file.tar -C /to/location/ --force-local --no-same-owner

Update Certificates

This will update certificates. After you put a certificate in /usr/local/share/ca-certificates/

update-ca-certificates

PipeStatus

somecommand
RETURN_CODE=${PIPESTATUS[0]}

Postgres: Misc

Here are some misc things you can do in postgres 9.4.

Print to Console:

 raise notice 'VALUE: %', value;

Json Build Object:

 json_build_object('name', value, 'name2': value2)

Json Array:

 json_agg(data)

Extract Month:
You can also extract year, day, etc.

 extract(month from date_column)

Json Querying:
You can extract a field from a json field. You can do:

 rec->>'id'
rec#>'{field, sub_field,value}'
rec->'object'

Row To Json:

 SELECT row_to_json(t)
FROM mytable t

Update With FROM:

 UPDATE mytable l
SET old_val_id=sub.new_val_id
FROM (
     SELECT l2.id, s.id as new_val_id
     FROM mytable l2
          INNER JOIN mysupertable s ON s.id=l2.old_val_id
) sub
WHERE sub.id=l.id;

Inline Script:

 DO $do$
DECLARE id integer;
BEGIN      

END $do$ LANGUAGE plpgsql;

If:

 IF CONDITION THEN
END IF;

Let’s say you have a json field and in that field you have a field which tells you what key you should select for certain data. It could be customizable from the user entering data. To dynamically select that field you can do the below.

jsonField->((to_json((jsonField->'key'->'sub_key'))#>>'{}')::text)

Upsert:
Sometimes we want to perform update if the record exists or an insert if it doesn’t. Most of the time we write a function that deals with this on a record by record basis. But what if we want to do a batch update or insert. What do we do then. We perform an upsert. See below. We write the update and return the result in the remainder of the query  we check if upsert gave us anything. If it didn’t we perform the insert.

WITH upsert AS (UPDATE MyTable mt
            SET column_name = sub.new_value
            FROM (SELECT mot.id, mot.new_value 
                 FROM MyOtherTable mot
                 ) sub
            WHERE sub.id=mt.related_id 
RETURNING *) 
INSERT INTO MyTable (id, column_name, related_id) 
SELECT ?, ?, ? 
WHERE NOT EXISTS (SELECT * FROM upsert)

Regex Substring:
There are different ways of using regex. This is one way.

substring(column_name from '([0-9]{4}-[0-9]{2}-[0-9]{2})')

PGSQL Loop:
This is one way to loop using pgsql.

DO $do$
DECLARE rec RECORD;
BEGIN
	FOR rec IN SELECT * FROM MyTable
	LOOP
		--We can then use rec like rec.colum_name
	END LOOP;
END $do$ LANGUAGE plpgsql;

Milliseconds to timestamp:

This will return 2017-08-17 21:26:04

select to_timestamp(1503005165);