Showing posts with label SQLite Database. Show all posts
Showing posts with label SQLite Database. Show all posts

Monday, January 19, 2015

How to select only duplicate records?

This is a guideline to select:

First ways:
 
Select State FROM Area
GROUP BY State
Having COUNT(*) > 1
 
Second ways:
 
 
SELECT DISTINCT a1.State
FROM AREA a1
JOIN AREA a2
  ON a1.AreaId != a2.AreaId  -- assume there is a Key to join on
  AND a1.State = a2.State    -- and such that different Areas with same State

Tuesday, August 19, 2014

Insert all values of a table into another table in SQL

The insert statement actually has a syntax for doing just that. It's a lot easier if you specify the column names rather than selecting "*" though:
 

INSERT INTO new_table (Foo, Bar, Fizz, Buzz) SELECT Foo, Bar, Fizz, Buzz FROM initial_table -- optionally WHERE ...

 
 
I'd better clarify this because for some reason this post is getting a few down-votes.
The INSERT INTO ... SELECT FROM syntax is for when the table you're inserting into ("new_table" in my example above) already exists. As others have said, the SELECT ... INTO syntax is for when you want to create the new table as part of the command.
You didn't specify whether the new table needs to be created as part of the command, so INSERT INTO ... SELECT FROM should be fine if your destination table already exists.

Friday, March 22, 2013

SQLite tutorial

This is SQLite tutorial. It covers the SQLite database engine, sqlite3 command line tool and the SQL language covered by the database engine.

Table of contents

SQLite

SQLite is an embedded relational database engine. Its developers call it a self-contained, serverless, zero-configuration and transactional SQL database engine.

Views, triggers, transactions

In this part of the SQLite tutorial, we will mention views, triggers and transactions.

Views

A view is a specific look on data in from one or more tables. It can arrange data in some specific order, higlight or hide some data. A view consists of a stored query accessible as a virtual table composed of the result set of a query. Unlike ordinary tables a view does not form part of the physical schema. It is a dynamic, virtual table computed or collated from data in the database.
In the next example, we create a simple view.
sqlite> SELECT * FROM Cars;
Id          Name        Cost      
----------  ----------  ----------
1           Audi        52642     
2           Mercedes    57127     
3           Skoda       9000      
4           Volvo       29000     
5           Bentley     350000    
6           Citroen     21000     
7           Hummer      41400     
8           Volkswagen  21600  
This is our data, upon which we create the view.
sqlite> CREATE VIEW CheapCars AS SELECT Name FROM Cars WHERE Cost < 30000;
sqlite> SELECT * FROM CheapCars;
Name      
----------
Skoda     
Volvo     
Citroen   
Volkswagen
The CREATE VIEW statement is used to create a view.
sqlite> .tables
Books         CheapCars     Friends       Names         Reservations
Cars          Customers     Log           Orders        Testing     
sqlite> DROP VIEW CheapCars;
sqlite> .tables
Books         Customers     Log           Orders        Testing     
Cars          Friends       Names         Reservations
Technically a view is a virtual table. So we can list all views with a .tables command. To remove a view, we use the DROP VIEW SQL statement.

Triggers

Triggers are database operations that are automatically performed when a specified database event occurs.
In the following example, we will use the Friends table and create a new Log table.
sqlite> CREATE TABLE Log(Id INTEGER PRIMARY KEY, OldName TEXT, 
   ...> NewName TEXT, Date TEXT);
The Log table has a column for the old name and for the new name of a friend. It also has a column for a timestamp.
CREATE TRIGGER mytrigger UPDATE OF Name ON Friends
BEGIN
INSERT INTO Log(OldName, NewName, Date) VALUES (old.Name, new.Name, datetime('now'));
END;
We create a trigger called mytrigger with the CREATE TRIGGER statement. This trigger will launch a INSERT statement whenever we update the name column of the Friends table. The INSERT statement will insert the old name, the new name and the time stamp into the Log table. The old and new are references to the row being modified.
sqlite> SELECT * FROM Friends;
Id          Name        Sex       
----------  ----------  ----------
1           Jane        F         
2           Thomas      M         
3           Franklin    M         
4           Elisabeth   F         
5           Mary        F         
6           Lucy        F         
7           Jack        M  
This is our data.
Next, we are going to update one row of the Friends table.
sqlite> UPDATE Friends SET Name='Frank' WHERE Id=3;
We update the third row of the table. The trigger is launched.
sqlite> SELECT * FROM Log;
Id          OldName     NewName     Date               
----------  ----------  ----------  -------------------
1           Franklin    Frank       2013-01-09 23:38:29
We check the Log table. This log confirms the update operation we performed.

Transactions

A transaction is an atomic unit of database operations against the data in one or more databases. The effects of all the SQL statements in a transaction can be either all committed to the database or all rolled back.
In SQLite, any command other than the SELECT will start an implicit transaction. Manual transactions are started with the BEGIN TRANSACTION statement and finished with the COMMIT or ROLLBACK statements.
BEGIN TRANSACTION;
CREATE TABLE Test(Id integer NOT NULL);
INSERT INTO Test VALUES(1);
INSERT INTO Test VALUES(2);
INSERT INTO Test VALUES(3);
INSERT INTO Test VALUES(NULL);
COMMIT;
Here we have a sample transaction. A transaction begins with BEGIN TRANSACTION and ends with COMMIT.
We have a NOT NULL constraint set on the Id column. Thus, the fourth insert will not succeed. SQLite does transactions specifically. For some errors, it reverts all changes. For others, it reverts only the last statement and leaves other changes intact. In our case, the table is created and the first three inserts are written into the table. The fourth one is not.
Say, we already had an empty table named Test. Executing the above transaction would fail completely. No changes would be written. If we changed the CREATE TABLE statement into CREATE TABLE IF NOT EXISTS, the first three statements would execute.
BEGIN TRANSACTION;
CREATE TABLE IF NOT EXISTS Test(Id integer NOT NULL);
INSERT INTO Test VALUES(1);
INSERT INTO Test VALUES(2);
INSERT INTO Test VALUES(3);
INSERT INTO Test VALUES(NULL);
ROLLBACK;
A transaction can end with a COMMIT or a ROLLBACK statement. The ROLLBACK reverts all changes.
In this part of the SQLite tutorial, we have worked with views, triggers and transactions in SQLite.

SQLite functions

In this part of the SQLite tutorial, we will cover SQLite built-in functions. There are three types of functions in SQLite database. Core, aggregate and date & time functions.
We will cover some functions from each group of SQLite functions.

Core functions

In this group we have various diverse functios. Some are numerical functions, some work with text. Others do some very specific things.
sqlite> SELECT sqlite_version() AS 'SQLite Version';
SQLite Version
--------------
3.7.15.1  
The sqlite_version() function returns the version of the SQLite library.
sqlite> SELECT random() AS Random;
Random             
-------------------
1056892254869386643   
The random() function returns a pseudo-random integer between -9223372036854775808 and +9223372036854775807.
sqlite> SELECT abs(11), abs(-5), abs(0), abs(NULL);
abs(11)             abs(-5)      abs(0)      abs(NULL) 
------------------  -----------  ----------  ----------
11                  5            0           NULL  
The abs() function returns the absolute value of the numeric argument.
sqlite> SELECT max(Cost), min(Cost) FROM Cars;
max(Cost)    min(Cost)  
-----------  -----------
350000       9000  
In our example, the max() and min() functions return the most and the least expensive cars from the Cars table.
sqlite> .width 18
sqlite> SELECT upper(Name) AS 'Names in capitals' FROM Friends;
Names in capitals 
------------------
JANE              
THOMAS            
FRANK             
ELISABETH         
MARY              
LUCY              
JACK  
The upper() function converts characters into upper-case letters.
sqlite> SELECT lower(Name) AS 'Names in lowercase' FROM Friends
   ...> WHERE Id IN (1, 2, 3);
Names in lowercase
------------------
jane              
thomas            
frank  
With the lower() function we change the names of first three rows into lower-case letters.
sqlite> SELECT length('ZetCode');
length('ZetCode') 
------------------
7 
The length() function returns the length of a string.
sqlite> SELECT total_changes() AS 'Total changes';
Total changes
-------------
3    
The total_changes() function returns the number of row changes caused by INSERT, UPDATE or DELETE statements since the current database connection was opened. In the current database connection, I have done three INSERT statements, so the total changes is equal to three.
sqlite> .width 5
sqlite> SELECT sqlite_compileoption_used('SQLITE_DEFAULT_FOREIGN_KEYS') AS 'FK';
FK   
-----
0  
The sqlite_compileoption_used() function returns a boolean value, depending on whether or not that option was used during the build. In our case we check if the FOREIGN KEY constraint is enforced by default. The function returns 0, which means that the constraint is not enforced by default. We use the PRAGMA statement to change it. (PRAGMA foreign_keys = 1;)
sqlite> SELECT typeof(12), typeof('ZetCode'), typeof(33.2), typeof(NULL), 
   ...> typeof(x'345edb');
typeof(12)    typeof('ZetCode')   typeof(33.2)  typeof(NULL)  typeof(x'345edb')
------------  ------------------  ------------  ------------  -----------------
integer       text                real          null          blob  
The typeof() function returns the data type of the argument.

Aggregate funcions

With aggregate functions, we get some statistical data.
Let's recap, what we have in the Cars table.
sqlite> SELECT * FROM Cars;
Id          Name        Cost      
----------  ----------  ----------
1           Audi        52642     
2           Mercedes    57127     
3           Skoda       9000      
4           Volvo       29000     
5           Bentley     350000    
6           Citroen     21000     
7           Hummer      41400     
8           Volkswagen  21600   
Notice, that there are no duplicate records.
sqlite> SELECT count(*) AS '# of cars' FROM Cars;
# of cars 
----------
8     
The count() function returns the number of rows in the table. In our table, we have eight cars. Assuming, there are no duplicates.
In the Orders table, we have duplicate records of customers.
sqlite> SELECT * FROM Orders;
Id          OrderPrice  Customer  
----------  ----------  ----------
1           1200        Williamson
2           200         Robertson 
3           40          Robertson 
4           1640        Smith     
5           100         Robertson 
6           50          Williamson
7           150         Smith     
8           250         Smith     
9           840         Brown     
10          440         Black     
11          20          Brown    
Logically, each customer can make multiple orders. How do we count the number of orders and how do we count the number of customers?
sqlite> SELECT count(Customer) AS '# of orders'  FROM Orders;
# of orders
-----------
11   
This SQL statement returns the number of orders. To calculate the number of unique customers, we have to utilize the DISTINCT clause.
sqlite> SELECT count(DISTINCT Customer) AS '# of customers' FROM Orders;
# of customers
--------------
5   
We have 5 customers in our Orders table. They made 11 orders.
Next we are going to demonstrate the difference between the count(*) and count(ColumnName). These function usages differ in the way, how they handle NULL values.
sqlite> .nullvalue NULL
First, we change how sqlite3 shows NULL values. By default, the NULL value is shown as empty string.
sqlite> CREATE TABLE Testing(Id INTEGER);
sqlite> INSERT INTO Testing VALUES(1);
sqlite> INSERT INTO Testing VALUES(2);
sqlite> INSERT INTO Testing VALUES(3);
sqlite> INSERT INTO Testing VALUES(NULL);
sqlite> INSERT INTO Testing VALUES(NULL);
sqlite> SELECT * FROM Testing;
Id          
------------
1           
2           
3           
NULL        
NULL 
Here we create table Testing with 3 numerical and 2 NULL values.
sqlite> SELECT count(*) AS '# of rows' FROM Testing;
# of rows 
----------
5  
The count(*) returns the number of rows in the table. It takes NULL values into account.
sqlite> SELECT count(Id) AS '# of non NULL values' FROM Testing;
# of non NULL values
--------------------
3 
The count(Id) counts only non NULL values.
sqlite> SELECT avg(Cost) AS 'Average price' FROM Cars;
Average price     
------------------
72721.125  
The avg() function returns the average value of all non NULL records. In our example, we show the average price of the car in the Cars table.
Finally, we mention the sum() function. It does a summation of all non NULL values.
sqlite> SELECT sum(OrderPrice) AS Sum FROM Orders;
Sum     
--------
4930   
Here we count the sum of all orders made by our customers.

Date and time funcions

SQLite has functions for working with date and time. With these functions we can use various time strings, modifiers and formats.
sqlite> .header OFF
sqlite> SELECT date('now');
2013-01-09    
The date() function with the now string returns the current date.
sqlite> SELECT datetime('now');
2013-01-09 13:01:0
The datetime() function returns the current date and time.
sqlite> SELECT time('now');
13:23:21  
The time() function gives the current time.
sqlite> SELECT time(), time('now');
13:50:12    13:50:12   
sqlite> SELECT date('now'), date();
2013-01-09   2013-01-09
The now string can be omitted.

The first parameter of the date(), time() and datetime() functions is the time string. It can be followed by one or more modifiers.
sqlite> SELECT date('now', '2 months');
2013-03-09 
In this example, '2 months' is a modifier. It adds two months to the current date. So the function returns the date two months from today.
sqlite> SELECT date('now', '-55 days');
2012-11-15
Negative modifiers can be also used. In this example, we extract 55 days from today.
sqlite> SELECT date('now', 'start of year');
2013-01-01  
Using the start of year modifier, we get the date of the start of the year, e.g. the January 1st.
sqlite> SELECT datetime('now', 'start of day');
2013-01-09 00:00:00  
With the help of the start of day modifier, we get the beginning of the current day.
sqlite> SELECT date('now', 'weekday 6');
2013-01-12 
The weekday modifier advances to the next date, where Sunday is 0, Monday 1, ..., Saturday 6. In this example, we get the date of the nearest Saturday.
The modifiers can be combined.
sqlite> SELECT date('now', 'start of year', '10 months', 'weekday 4');
2013-11-07
This SQL statement returns the first Thursday of the November for the current year. In this example, we used three modifiers. start of year, +x months and weekday x. The now time string gives the current date. The start of year shifts the date backwards to the beginning of the year. The10 months adds 10 months to the current month (January). Finally, the weekday 4 modifier advances the date forward to the first Thursday.

The strftime() function returns the date and time formatted according to the format string specified as the first argument. The second parameter is the time string. It can be followed by one or more modifiers.
sqlite> SELECT strftime('%d-%m-%Y');
09-01-2013 
We can use the the strftime() function to return a date in a different format.
sqlite> SELECT 'Current day: ' || strftime('%d');
Current day: 09  
This SQL statement returns the current day of the month. We used the strftime() function.
sqlite> SELECT 'Days to XMas: ' || (strftime('%j', '2013-12-24') -
   ...> strftime('%j', 'now'));
Days to XMas: 349 
Here we have computed the number of days till Christmas. The %j modifier gives the day of the year for the time string.
In this part of the SQLite tutorial, we worked with the built-in SQLite functions.

Joining tables

In this part of the SQLite tutorial, we will join tables in SQLite.
The real power and benefits from relational databases come from joining tables. The SQL JOIN clause combines records from two or more tables in a database. There are basically two types of joins. INNER and OUTER.
In this part of the tutorial, we will work with Customers and Reservations tables.
sqlite> SELECT * FROM Customers;
CustomerId  Name       
----------  -----------
1           Paul Novak 
2           Terry Neils
3           Jack Fonda 
4           Tom Willis 
Values from the Customers table.
sqlite> SELECT * FROM Reservations;
Id  CustomerId  Day       
--  ----------  ----------
1   1           2009-22-11
2   2           2009-28-11
3   2           2009-29-11
4   1           2009-29-11
5   3           2009-02-12
Values from the Reservations tables.

Inner joins

The inner join is the most common type of joins. It is the default join also. The inner join selects only those records from database tables that have matching values. We have three types of INNER JOINS. INNER JOIN, NATURAL INNER JOIN and CROSS INNER JOIN. The INNER keyword can be omitted.

INNER JOIN

sqlite> SELECT Name, Day FROM Customers AS C JOIN Reservations
   ...> AS R ON C.CustomerId=R.CustomerId;
Name         Day        
-----------  -----------
Paul Novak   2009-22-11 
Terry Neils  2009-28-11 
Terry Neils  2009-29-11 
Paul Novak   2009-29-11 
Jack Fonda   2009-02-12 
In this SELECT statement, we have selected all customers, that have made some reservations. Note, that we have omitted the INNER keyword.
The statement is equivalent to the following one:
sqlite> SELECT Name, Day FROM Customers, Reservations
   ...> WHERE Customers.CustomerId = Reservations.CustomerId;
Name        Day        
----------  -----------
Paul Novak  2009-22-11 
Terry Neil  2009-28-11 
Terry Neil  2009-29-11 
Paul Novak  2009-29-11 
Jack Fonda  2009-02-12
We get the same data.

NATURAL INNER JOIN

The NATURAL INNER JOIN automatically uses all the matching column names for the join. In our tables, we have a column named CustomerId in both tables.
sqlite> SELECT Name, Day FROM Customers NATURAL JOIN Reservations;
Name         Day       
-----------  ----------
Paul Novak   2009-22-11
Terry Neils  2009-28-11
Terry Neils  2009-29-11
Paul Novak   2009-29-11
Jack Fonda   2009-02-12
We get the same data. The SQL statement is less verbose.

CROSS INNER JOIN

The CROSS INNER JOIN combines all records from one table with all records from another table. This type of join has little practical value. It is also called a cartesian product of records.
sqlite> SELECT Name, Day FROM Customers CROSS JOIN Reservations;
Name         Day       
-----------  ----------
Paul Novak   2009-22-11
Paul Novak   2009-28-11
Paul Novak   2009-29-11
Paul Novak   2009-29-11
Paul Novak   2009-02-12
Terry Neils  2009-22-11
Terry Neils  2009-28-11
Terry Neils  2009-29-11
Terry Neils  2009-29-11
Terry Neils  2009-02-12
...
The same result can be achieved with the following SQL statement:
sqlite> SELECT Name, Day FROM Customers, Reservations;

Outer joins

An outer join does not require each record in the two joined tables to have a matching record. There are three types of outer joins. Left outer joins, right outer joins, and full outer joins. SQLite only supports left outer joins.

LEFT OUTER JOIN

The LEFT OUTER JOIN returns all values from the left table, even if there is no match with the right table. It such rows, there will be NULL values. In other words, left outer join returns all the values from the left table, plus matched values from the right table. Note, that the OUTER keyword can be omitted.
sqlite> SELECT Name, Day FROM Customers LEFT JOIN Reservations
   ...> ON Customers.CustomerId = Reservations.CustomerId;
Name         Day        
-----------  -----------
Paul Novak   2009-22-11 
Paul Novak   2009-29-11 
Terry Neils  2009-28-11 
Terry Neils  2009-29-11 
Jack Fonda   2009-02-12 
Tom Willis   NULL  
Here we have all customers with their reservations, plus a customer, who has no reservation. There is NULL value in his row.
We can use the USING keyword to achieve the same result. The SQL statement will be less verbose.
sqlite> SELECT Name, Day FROM Customers LEFT JOIN Reservations
   ...> USING (CustomerId);
Name         Day        
-----------  -----------
Paul Novak   2009-22-11 
Paul Novak   2009-29-11 
Terry Neils  2009-28-11 
Terry Neils  2009-29-11 
Jack Fonda   2009-02-12 
Tom Willis   NULL
Same result, with shorter SQL statement.

NATURAL LEFT OUTER JOIN

The NATURAL LEFT OUTER JOIN automatically uses all the matching column names for the join.
sqlite> SELECT Name, Day FROM Customers NATURAL LEFT OUTER JOIN Reservations;
Name         Day       
-----------  ----------
Paul Novak   2009-22-11
Paul Novak   2009-29-11
Terry Neils  2009-28-11
Terry Neils  2009-29-11
Jack Fonda   2009-02-12
Tom Willis   NULL  
Same result, but with fewer key strokes.
In this part of the SQLite tutorial, we were joining tables.