WiFi Hotspot greyed – System Policy prohibits use as a Hotspot [Solved]

Sometimes when you open GNOME settings to the WiFi page, the “create hotspot” option is initially greyed out. This can be solved easily by switching to any other settings page and coming straight back makes the option available.

Steps to solve this

  1. Open GNOME Settings
  2. Try to activate hotspot, it is greyed out
  3. Switch back to “Network” or “Bluetooth” or any setting page and go back to WiFi settings
  4. Activate hotpot” menu option should be available now

This works in all Linux distros that use GNOME (Ubuntu, Manjaro, Fedora, etc..,)

Learn Candlestick analysis

Candlestick charts provide visual insight to current market psychology. A candlestick displays the open, high, low, and closing prices in a format similar to a modern-day bar-chart, but in a manner that extenuates the relationship between the opening and closing prices. Candlesticks don’t involve any calculations. Each candlestick represents one period (e.g., minute, hour, day) of data.

One cannot ignore that investor’s psychologically driven forces of fear; greed and hope greatly influence the stock prices. The overall market psychology can be tracked through candlestick analysis. More than just a method of pattern recognition, candlestick analysis shows the interaction between buyers and sellers. A green candlestick indicates the opening price of the session is below the closing price, and a red candlestick shows the opening price of the session is above the closing price. The shadow at top and bottom indicates the high and low for the session.

candlestick

Japanese candlesticks offer a quick picture into the psychology of short term trading, studying the effect, not the cause. Therefore if you combine candlestick analysis with other technical analysis tools, candlestick pattern analysis can be a very useful way to select entry and exit points.

  • Single candlestick patterns
    • Doji
    • Spinning Top
    • Marubuzo
      • Bullish Marubuzo
      • Bearish Marubuzo
    • Hammer
    • Hanging Man
    • Shooting Star
  • Multiple candlestick patterns
    • Engulfing Pattern
      • Bullish Engulfing
      • Bearish Engulfing
    • Piercing Pattern
    • Dark Cloud Cover
    • Harami Pattern
      • Bullish Harami
      • Bearish Harami
    • Candle Gaps
    • Morning Star
    • Evening Star
    • Three White Soldiers
    • Three Black Crows

Descriptive Statistics in R

In a series of learning data analysis using R, Let’s see different methods to perform descriptive statistics in R. This includes measures of central tendency, variability, and distribution shape for continuous variable.

For this tutorial, we shall use built-in dataset mtcars. This dataset consists of 32 observations and 11 variables. We shall use three variables alone for calculating descriptive statistics.

Refine the data

> data("mtcars")
> data<-c("mpg","hp","wt")
> head(mtcars[data])
                   mpg  hp    wt
Mazda RX4         21.0 110 2.620
Mazda RX4 Wag     21.0 110 2.875
Datsun 710        22.8  93 2.320
Hornet 4 Drive    21.4 110 3.215
Hornet Sportabout 18.7 175 3.440
Valiant           18.1 105 3.460
> data("mtcars")

The base R installation has summary() function which shall be used to obtain descriptive statistics.

Example for descriptive statistics

> summary(mtcars[data])
      mpg              hp              wt       
 Min.   :10.40   Min.   : 52.0   Min.   :1.513  
 1st Qu.:15.43   1st Qu.: 96.5   1st Qu.:2.581  
 Median :19.20   Median :123.0   Median :3.325  
 Mean   :20.09   Mean   :146.7   Mean   :3.217  
 3rd Qu.:22.80   3rd Qu.:180.0   3rd Qu.:3.610  
 Max.   :33.90   Max.   :335.0   Max.   :5.424

The summary() function provides the minimum, maximum, quartiles, and the mean for numerical variables and frequencies for factors and logical vectors. The above results doesn’t include Standard deviation, Skewness, Kurtosis and Variance. What if you need to calculate these statistics?. For this you may use stat.desc function in pastecs package.

> install.packages("pastecs")
> library(pastecs)
> stat.desc(mtcars[data])
                     mpg           hp          wt
nbr.val       32.0000000   32.0000000  32.0000000
nbr.null       0.0000000    0.0000000   0.0000000
nbr.na         0.0000000    0.0000000   0.0000000
min           10.4000000   52.0000000   1.5130000
max           33.9000000  335.0000000   5.4240000
range         23.5000000  283.0000000   3.9110000
sum          642.9000000 4694.0000000 102.9520000
median        19.2000000  123.0000000   3.3250000
mean          20.0906250  146.6875000   3.2172500
SE.mean        1.0654240   12.1203173   0.1729685
CI.mean.0.95   2.1729465   24.7195501   0.3527715
var           36.3241028 4700.8669355   0.9573790
std.dev        6.0269481   68.5628685   0.9784574
coef.var       0.2999881    0.4674077   0.3041285

There are many other packages that are available you may try describe() function in psych package. and let me know which is your preferable function/package for calculating descriptive statistics.

Importing data from SPSS in R

SPSS is one of the popular statistical package used for data analysis. But remember you have to pay a huge amount to buy this software. The SPSS datasets can be imported into R using read.spss() function in foreign package .

Alternatively, you can use the spss.get() function in the Hmisc package .

spss.get() is a wrapper function that automatically sets many parameters of read.spss() for you, making the transfer easier and more consistent with what data analysts expect as a result.

First, download and install the Hmisc package, foreign package is already installed by default in R

install.packages("Hmisc")

Then use the following code to import the data:

library(Hmisc)
dataframe <- spss.get("sample.sav", use.value.labels=TRUE)

In this code, sample.sav is the SPSS data file to be imported, use.value.
labels=TRUE tells the function to convert variables with value labels into R factors with those same levels, and dataframe is the resulting R data frame.

 

Useful functions for working with data objects in R

We have seen different data structures in R. Now let’s see some useful functions for working with these data objects.

Functions Purpose
length(x) Number of elements/components.
dim(x) Dimensions of x.
str(x) Structure of x.
class(x) Class or type of x.
mode(x) How x is stored.
names(x) Names of components in x.
c(x1, x2,...) Combines x1,x2 into a vector.
cbind(x1, x2, ...) Combines x1,x2 as columns.
rbind(x1, x2, ...) Combines x1,x2 as rows.
x Prints the x.
head(x) Lists the first part of the x.
tail(x) Lists the last part of the x.
ls() Lists current objects.
rm(x1, x2, ...) Deletes one or more objects. The statement
rm(list = ls()) will remove most objects
from the working environment.
y <- edit(x) Edits x and saves as y.
fix(x) Edits in place.

If I have missed any functions, Let me know in the comment section below.

Data structures in R – Part 2

R has wide options for holding data, such as scalars, vectors, matrices, arrays, data frames, and lists. In Data structures in R – Part 1 we have seen scalars, vectors, matrices, arrays. Now let’s see data frames and lists.

Data frames

A data frame is more is a table or a two-dimensional array-like structure in which each column contains values of one variable and each row contains one set of values from each column.  Here, different columns can contain different modes of data (numeric, character, etc.). It’s similar to the datasets that we see in IBM SPSS, SAS and Stata. Data frames are the most common data structure that is used within R.

Characteristics of a data frame

  1. Column names should not be empty.
  2. Row names should be unique.
  3. Data stored in a data frame can be of numeric, factor or character type.
  4. Each column should contain the same number of data items.

Let’s see some example,

> marklist<-data.frame( + rollno = c(1001:1006), + name = c("Abdul","Balu","Charlie","Daniel","Elisa","Fathima"), + marks = c(87,91,66,57,83,72) + ) > marklist
  rollno    name marks
1   1001   Abdul    87
2   1002    Balu    91
3   1003 Charlie    66
4   1004  Daniel    57
5   1005   Elisa    83
6   1006 Fathima    72

In the above example, you can observe that each column must have only one data type but you can have different columns inside the data frame with the different data type.

We can subscript data frame like the way we subscript matrices. Let’s see this with an example with the above used marklist dataset.

> marklist[1,3]
[1] 87
> marklist[1:3]
  rollno    name marks
1   1001   Abdul    87
2   1002    Balu    91
3   1003 Charlie    66
4   1004  Daniel    57
5   1005   Elisa    83
6   1006 Fathima    72
> marklist[c(1,3)]
  rollno marks
1   1001    87
2   1002    91
3   1003    66
4   1004    57
5   1005    83
6   1006    72
> marklist[c("rollno","marks")]
  rollno marks
1   1001    87
2   1002    91
3   1003    66
4   1004    57
5   1005    83
6   1006    72
> marklist$name
[1] "Abdul"   "Balu"    "Charlie" "Daniel" 
[5] "Elisa"   "Fathima"

Factors

Factors are used to categorize the data and store it as levels. They can store both strings and integers. This is useful in the columns which have a limited number of unique values. For example, Male, Female, Neutral and True, False etc. They are useful in data analysis for statistical modelling. Factors are created using the factor () function by taking a vector as input.

We will see more about factors practically when we discuss about statistical methods.

Data structures in R – Part 1

R has wide options for holding data, such as scalars, vectors, matrices, arrays, data frames, and lists. Let’s look at each structure in this post.

Scalars

Scalars are one-element vectors. These are used to hold constants.

Example

a <- 1
b < "Phone"
c <- TRUE

Vectors

Vectors are one-dimensional arrays that hold numbers, characters, or logical data. The combine function c() is used to form a vector. Vectors can hold only one data type you can mix numbers with characters. Let’s look at some example

Numeric vector

a <- c(2,10,-5,15)

Character vector

b <- c("Male", "Female", "Neutral")

Logical vector

c <- c(TRUE, FALSE, FALSE, TRUE)

To refer an elements of a vector you can use square brackets. For example,

a<-c(2,4,6,8,10,12,14,16,18,20)
> a[6]
[1] 12
> a[3:6]
[1] 6 8 10 12
> a[c(1,7)]
[1] 2 14

Matrices

A matrix is a two-dimensional array where each element has the same data type. Matrices are created with the matrix function. The syntax for matric function is

a <- matrix(vector, nrow=number_of_rows, ncol=number_of_columns,
byrow=logical_value, dimnames=list( char_vector_rownames, char_vector_colnames))

where vector contains the elements for the matrix, nrow and ncol specify the row and column dimensions, and dimnames contains optional row and column labels stored in character vectors. The option byrow indicates whether the matrix should be filled in by row ( byrow=TRUE ) or by column ( byrow=FALSE ). The default is by column. The following listing demonstrates the matrix function.

Let’s see some examples for matrices now

Creating a 5×2 matrix

> a<-matrix(1:10, nrow=5,ncol=2)
> a
     [,1] [,2]
[1,]    1    6
[2,]    2    7
[3,]    3    8
[4,]    4    9
[5,]    5   10

Let’s create a 2x2 matrix with row and column label

> cells <- c(2,8,12,16)
> r <- c("A1","A2")
> c <- c("X1","X2")
> b<-matrix(cells,nrow = 2, ncol = 2, byrow = TRUE,dimnames = list(r,c))
> b
   X1 X2
A1  2  8
A2 12 16

In the above example, a matrix was created byrow = TRUE, try the same argument with FALSE and see the difference.

Subscripts in matrix

You can also subscript matrix using square brackets

> x<-matrix(11:20, nrow=2)
> x
     [,1] [,2] [,3] [,4] [,5]
[1,]   11   13   15   17   19
[2,]   12   14   16   18   20
> m <-x[,3]
> m
[1] 15 16
> n <-x[1,4]
> n
[1] 17
> o <-x[2,c(3,4,5)]
> o
[1] 16 18 20

First, we created a 2x5 matrix, then we subscript the matrix with square brackets mentioning the column number and row number.

Arrays

Arrays are similar to matrices, the difference is this can have more than two dimensions. If we create an array of dimension (2, 3, 4) then it creates 4 rectangular matrices each with 2 rows and 3 columns. Arrays can store only data type. This can be created with array function. The syntax for the function is

array<-array(vector, dimentions, dimnames)

Here vector contains the data for the array, dimensions is the numeric vector giving maximal index for each dimension and dimnames is an optional list of dimension labels. This is useful in programming new statistical methods.

Let’s see this with the following examples,

> column <- c("COL1","COL2","COL3") > row <- c("ROW1","ROW2","ROW3") > matrix <- c("Matrix1","Matrix2") > a <- array(1:24,c(3,3,2),dimnames = list(column,row,matrix)) > a
, , Matrix1

     ROW1 ROW2 ROW3
COL1    1    4    7
COL2    2    5    8
COL3    3    6    9

, , Matrix2

     ROW1 ROW2 ROW3
COL1   10   13   16
COL2   11   14   17
COL3   12   15   18

Keep reading about data structures. Data structures in R – Part 2

Linux Beginner’s Guide Part 6

The ifconfig command is used to view the current network configuration for ethernet cards and iwconfig displays the network configuration for wireless cards.

ifconfig [OPTIONS]

iwconfig [OPTIONS]

[[email protected] ~]$ ifconfig
enp2s0f1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether d8:c4:97:41:27:de  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 38  bytes 4503 (4.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 38  bytes 4503 (4.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:55:6e:ed  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.43.182  netmask 255.255.255.0  broadcast 192.168.43.255
        inet6 2409:4072:610a:f3b:cb9c:bd9f:ff6e:a23c  prefixlen 64  scopeid 0x0
        inet6 fe80::9a9d:5958:dbf1:6836  prefixlen 64  scopeid 0x20
        ether 98:22:ef:b4:70:e7  txqueuelen 1000  (Ethernet)
        RX packets 265050  bytes 335273482 (319.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 109678  bytes 16079690 (15.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[[email protected] ~]$

The ifconfig and iwconfig can be used to make temporary changes in the network interfaces.

Processes

Running any kind of application/command/game and so on, creates something that is called process. The process runs with the privileges with the user who created it, this way the process is controlled and limited to run only under certain capabilities. The user who creates a process is the owner if it and it can not be controlled by other users with same or lower privileges. In our case, if the sysadmin user runs a process, another user with same or lower privilege level, can not control it but it can be controlled by the root user. The ps command is used to view the processes.

ps [OPTIONS]

The default output for ps command includes 4 columns and displays only the processes for the current terminal:

PID: process identifier, is a unique ID used to identify each process and to control it;

TTY: terminal name where the process is running, is used to identify a process if there are more that have same name;

TIME: total amount of processor time used by the process;

CMD: command that started the process;

To view the processes from the entire system ps -e command must be executed and ps -f displays more details for each process. The two commands above can be combined into one: ps -ef.

Updating the passwords in Linux

The passwd command is used to change the password. The users can only change their own password and the root user can change the password of any user.

passwd [OPTIONS] [USER]

If the passwd command is used without arguments, the terminal automatically starts the procedure for changing the password for current user.

If you want to see the details about the status of the password you must follow the following steps:

[[email protected] ~]$ sudo -i
[sudo] password for sulthan: 
[[email protected] ~]# passwd -S sulthan
sulthan PS 2020-08-08 0 99999 7 -1 (Password set, SHA512 crypt.)
[[email protected] ~]#

1st column: username – sulthan

2nd column: password statusP = usable password; L = locked password; NP = no password;

3rd column: date – when the password was last change

4th column: minimum – the minimum number of days until the user can change the password

5th column: maximum – the maximum number of days until the password will expire

6th column: warn – the number of days before the password will expire and the user will be warned

7th column: inactive – the number of days after the password will expire and the user will remain active

8th column: info – details about the password status and encryption

Linux Beginner’s Guide Part 5

Installing Packages / Applications / Software in Linux

The dnf in fedora (apt-get or apt in Ubuntu/Debian based distributions) command is used to install new packages and to update/upgrade the operating system, it requires administrative access, so it must be used with sudo.

dnf [OPTIONS] [COMMAND]
apt-get [OPTIONS] [COMMAND] (Ubuntu)
apt [OPTIONS] [COMMAND] (Ubuntu)

The operating system must be updated on a regular basis, this way you will keep your data and integrity of the system safe. In order to update your system, run one of the following commands:

dnf update
apt-get update

dnf upgrade (updates the obsolete packages too)
apt-get upgrade

After the system is updated you can run dnf clean, to erase the files that were left after the packages installed.

apt-get autoremove (Ubuntu)
apt-get autoclean (Ubuntu)
apt-get clean (Ubuntu)

To search for a specific package/application/software in the repository (we will search for Gimp – a graphic software) use:

dnf search gimp (apt-get search gimp in Ubuntu)

and to install it run:

dnf install gimp (apt-get install gimp in Ubuntu).

And if you no longer need it:

dnf erase gimp (apt-get remove gimp in Ubuntu).

(use: dnf reinstall gimp in case you deleted it by mistake)

By default the Linux repository contains only packages that are considered stable, but we will see later in this tutorial, how to modify the repository and add more packages.

Shutdown command in Linux

The shutdown command is used to prepare the system to turn off in a safe way, all logged in users will be announced that the system will be turned off and in the last 5 minutes new logins are blocked.

shutdown [OPTIONS] [TIME] [MESSAGE]

Please note that the shutdown command requires administrative access, so you must login with the root user: su root

The argument [TIME] can accept the following options: now, hh:mm, +minutes

To see the current date/time and date/time format of your system run this command in the console: date

Do a few exercises

shudown 08:32

shutdown +3 “Bye Bye!”

To stop a shutdown command you must use: shutdown -c

Su users can also use this command in order to issue a restart:

reboot

shutdown -r now

halt

Make sure you save your work before you run commands like restart or shutdown.

Linux Beginner’s Guide Part 4

Filtering Input

grep command is a text filter that searches for the input and returns the text lines by a given pattern.

grep [OPTIONS] [PATTERN] [FILE]

In order to start the example, use the following command to get to the /Documents/ folder: cd ~/Documents

and then copy the file on which we will make tests, to the current directory: cp /etc/passwd . (note the dot, it’s important!)

next, run the grep command line this:

grep sysadmin passwd

The line that will be returned from the passwd file contains the pattern sysadmin.

Edit passwd file, add more lines that will contain the word sysadmin, then run the grep command and see what happens.

Regular Expressions

Regular expressions have two forms: basic and extended,

Basic regular expressions:

. any one single character

[ ] any one specified character

[^ ] not the specified character

* zero or more of the previous character

^ if the first character in a pattern, the pattern must be at the beginning of the line to match

$ if the last character in a pattern, the pattern must be at the end of the line to match

Extended regular expressions (must be used with grep command or grep -E command):

+ one or more of the previous pattern

? the preceding pattern is optional

{ } specify minimum, maximum or exact matches of a previous pattern

| alteration, a logical “or”

( ) used to create groups

Basic patterns

Regular expressions can be improved in order to match certain sequences of characters in a text.

Anchor characters

Anchor characters are used to mark and filter search result. To prevent the shell from wrong interpretation of the commands the word to be searched for must be placed between ‘ ‘, like in the example below:

grep ‘root’ passwd

How many times the root word is displayed in the output? Correct answer is 4 times.

Run the following command: grep ‘^root’ passwd ( ^ character ensures that the pattern appears at the beginning of each line)

In the next example run cat command to see the contents of passwd file: cat passwd

Run: grep ‘n$’ passwd command and see what will be displayed. $ character ensures that the patterns appears at end of each line.

For example you can run: grep ‘n…..n’ passwd. It will mark all words that starts with n character have any 5 letters inside and ends with n character. You can also try: grep ‘…’ passwd or grep ‘.a..’ passwd and see what will be displayed.

Let’s try to filter numbers: grep ‘[0-9]’ passwd. This command will mark all numbers from 0 to 9.

To negate all numbers from 0 to 9 use the following command: grep ‘[^0-9]’ passwd

Mark special characters: grep ‘[.]’ passwd

To mark 0(zero) or more occurrences of a character: grep ‘s*’ passwd

grep ‘n[bs]s*’ passwd

As we will see later in this tutorial, grep is a very important command and will ease your work a lot if used properly.

Using grep with firewall logs: grep -iE ‘(192.168.1.1|DROP|DPT=443|)’ firewall.log
This command will search in firewall.log and will mark the ip 192.168.1.1 or DROP or DPT=443 (destination port 443, port 443 is used by https).

tail or cat commands can be combined with grep or egrep. This way will be outputted only the lines with the requested keywords:

cat | grep -iE ‘(192.168.1.1|DROP|DPT=443|)’ firewall.log

tail | grep -iE ‘(192.168.1.1|DROP|DPT=443|)’ firewall.log

On other Linux distributions might work as well the following commands, for multiple grep commands combined with cat, tac or tail:

cat firewall.log | egrep -iE “192.168.1.1|DROP|DPT=443”

tac firewall.log | egrep -iE “192.168.1.1|DROP|DPT=443”

tail firewall.log | egrep -iE “192.168.1.1|DROP|DPT=443”