Tag: date

Shell: Misc Commands

In this tutorial I will show a few useful commands when working with linux shell.

Check Directory Exists:

if [ -d /opt/test/ ]; then
    echo 'Directory Exists'
fi

Check Directory Not Exists:

if [ ! -d /opt/test/ ]; then
    echo 'Directory Does Not Exist'
fi

Check File Exists:

if [ -f /opt/test/test.log ]; then
    echo 'File Exists'
fi

Check File Not Exists:

if [ ! -f /opt/test/test.log ]; then
    echo 'File Does Not Exist'
fi

Lowercase Variable:

val='TEXT'
echo "${val,,}"

Echo Variable:
This will print the value of “test”. Notice we use double quotes.

test='My Test Val'
echo "$test"

Echo String:

echo 'My test string'

Split:
This will split on the comma into an array list and then loop through it.

test='test 1,test 2'
split_test=(${test//,/ })

for val in "${split_test[@]}"
do
    echo $val
done

Date:
This will print the date in the format YYYY-MM-dd

my_date="$(date +Y-%m-%d)"
echo "$my_date"

Remove Space From Variable:

VAL='test string'
echo "${VAL//\ /}"

Increment Variable:

index=0
index=$((index+1))

Substring

VAL='test string'
echo "${VAL:4:4}"

If value is equal to

VAL='hello'
if [ "$VAL" == 'hello' ] ; then
    echo 'Is Equal'
fi

If with OR

VAL='hello'
if [ "$VAL" == 'hello' ] || [ "$VAL" != 'hi' ] ; then
    echo 'Is Hello'
fi

If Variable is Empty

VAL=''
if [ -z "$VAL" ] ; then
    echo 'Is Empty'
fi

Append to File

echo 'Hi' >> file_to_log_to.log

Write to File

echo 'Hi' > file_to_log_to.log

While Loop: Increment to 10

This will loop till the value is 9 then exit.

i=0
while [ $i -lt 10 ];
do
    echo "$i"
done

whoami

USER=$(whoami)

If Variable Contains Text

VAL='my Test String'
if [[ "${VAL,,}" == *"test"* ]] ; then
    echo "Found test"
fi

Color Coding

NoColor=$'\033[0m'
READ=$'\033[0;31m'
GREEN=$'\033[0;32m'
YELLOW=$'\033[1;33;40m'

printf "%s Variable Not Set %s\n" "${RED}" "${NoColor}"

Get Log to a logfile and console

SOME_COMMAND 2>&1 | tee -a "${LOG_FILE_PATH}"

Read a JSON config

JSON=$(cat "/path/to/json/file.json")
export MY_VAR=$(echo "${JSON}" | python -c 'import json,sys;obj=json.load(sys.stdin);print(obj["MyKey"])')

Extract tar to Folder

sudo tar -xvf /the/location/file.tar -C /to/location/ --force-local --no-same-owner

Update Certificates

This will update certificates. After you put a certificate in /usr/local/share/ca-certificates/

update-ca-certificates

PipeStatus

somecommand
RETURN_CODE=${PIPESTATUS[0]}

Python: Working with DateTimes

In this tutorial I will show you the different ways of working with dates and times in python. Which includes working with milliseconds. You should note this isn’t all available options just some that I have encountered over the years.

Install Python Packages:

Open cmd/terminal and if required navigate to your sites working folder. (note: if you are working in a virtual env you should ensure you source it first).

pip install python-dateutil

There are many different packages that we can use to work with date and times. You need to decide what is right for you.

dateutil:

The following will convert the date string you give it fast and easily. This gives you back the datetime object. Notice how we don’t need to pass it a date time format. To me this is very convenient.

from dateutil import parser

date_str = '2017-06-06'
date_time_str = '2017-06-07 12:34'
date_time_str_2 = '2017-06-07 12:34:46'
date_time_str_3 = '2017-06-07 12:34:42.234'

result = parser.parse(date_str)
print(result) #2017-06-06 00:00:00
result = parser.parse(date_time_str)
print(result) #2017-06-07 12:34:00
result = parser.parse(date_time_str_2)
print(result) #2017-06-07 12:34:46
result = parser.parse(date_time_str_3)
print(result) #2017-06-07 12:34:42.234000

datetime:

The following will convert the date string you give it fast and easily. This gives you back the datetime object. Notice how we need to pass the format of the datetime. If you don’t you will get an exception. This is a convenient way if you know the format before hand. But that might not always be the case.

import datetime

date_str = '2017-06-06'
date_time_str = '2017-06-07 12:34'
date_time_str_2 = '2017-06-07 12:34:46'
date_time_str_3 = '2017-06-07 12:34:42.234'

result = datetime.datetime.strptime(date_str, "%Y-%m-%d")
print(result) #2017-06-06 00:00:00
result = datetime.datetime.strptime(date_time_str, "%Y-%m-%d %H:%M")
print(result) #2017-06-07 12:34:00
result = datetime.datetime.strptime(date_time_str_2, "%Y-%m-%d %H:%M:%S")
print(result) #2017-06-07 12:34:46
result = datetime.datetime.strptime(date_time_str_3, "%Y-%m-%d %H:%M:%S.%f")
print(result) #2017-06-07 12:34:42.234000

The above all works however the following example will not. Why do you think this is?

import datetime

date_time_str = '2017-06-07 12:34:46'

try:
    datetime.datetime.strptime(date_time_str, "%Y-%m-%d %H:%M:%S")
except:
    pass #just for this example don't do this lol

The reason is because datetime expects the correct format to be supplied. We gave it hour minute second but not milliseconds. You will get the following exception (ValueError: unconverted data remains: .234)

Timestamps:

Sometimes we want to convert the date to unix (epoch) time or vise versa.

From Date:
from dateutil import parser
from datetime import timezone

date_time_str = '2017-06-07 17:34:42.234'
result = parser.parse(date_time_str)

timestamp = result.replace(tzinfo=timezone.utc).timestamp()
print(timestamp) #1496856882.234

This gives us the timestamp as a float as 1496856882.234.

From Timestamp:
from dateutil import parser
import datetime

timestamp = 1496856882.234

result = datetime.datetime.fromtimestamp(timestamp)
print(result) #2017-06-07 13:34:42.234000

result = datetime.datetime.utcfromtimestamp(timestamp)
print(result) #2017-06-07 17:34:42.234000

Get Date Parts:

If you want to get specific date parts such as the year, month, day, hour, etc.

import datetime
from dateutil import parser

result = parser.parse(date_time_str_3)
print(result) #2017-06-07 12:34:42.234000

year = result.year #2017
month = result.month #6
day = result.day #7
hour = result.hour #12
minute = result.minute #34
second = result.second #42
millisecond = result.microsecond #234000

Add To Date:

If you want to add time to a date.

import datetime
from dateutil import parser
from datetime import timezone, timedelta

date_time_str = '2017-06-07 17:34:42.234'
result = parser.parse(date_time_str)
print(result) #2017-06-07 17:34:42.234000

timestamp = result.replace(tzinfo=timezone.utc).timestamp()
print(timestamp) #1496856882.234

#Add 10 seconds to datetime
new_time = int((datetime.datetime.fromtimestamp(timestamp) + timedelta(milliseconds=10000)).timestamp() * 1000)
print(new_time) #1496856892234

As you can see you can 10 seconds has been added the datetime.

datetime strftime

from datetime import datetime

now = datetime.now()
datetime_str = now.strftime("%Y-%m-%d %H:%M:%S")
print(datetime_str)

datetime fromisoformat

from datetime import datetime

print(datetime.fromisoformat("2024-04-09 13:48:20"))