CS125 Project, Part 4

Due Thursday, October30, 2003

Step 4 of Your PDA

This week we shall write and run some SQL queries. It is suggested that you begin to follow the routine of loading your database from your load file, running queries or other commands from a file that contains those SQL statements, and then deleting your data so it doesn't clutter up the database all week and (worse) you don't forget and load the same tuples several times into your relations. Remember that SQL thinks of relations as bags, and so will happily let you insert the same tuple as many times as you ask it to. To clean out a relation R without deleting its schema, use command DELETE FROM R;.

Submission guidelines

You should submit a zip or tar file that expands to create a directory identified by your Postgres user name followed by "p4".  Please name the tar/zip file in the same way (followed by the appropriate file extension).  Your directory should contain four files named "a" through "d" containing queries and results as described below.

Deliverables

(a)
Write five queries on your PDA database, using the select-from-where construct of SQL. To receive full credit, all but perhaps one of your queries should exhibit some interesting feature of SQL: queries over more than one relation, or subqueries, for example. We suggest that you experiment with your SQL commands on a small database (e.g., your hand-created database), before running them on the large database that you loaded in PDA part 3. Initial debugging is much easier when you're operating on small amounts of data. Once you're confident that your queries are working, run them on your complete database. If you discover that most or all of your ``interesting'' queries return an empty answer on your large database, check whether you followed the instructions in Assignment #3 for generating data values that join properly. You may need to modify your data generator accordingly. Turn in a copy of all of your SQL queries, along with a script illustrating their execution. Your script should be sufficient to convince us that your commands run successfully. Please do not, however, turn in query results that are thousands (or hundreds of thousands) of lines long!  (Use the "LIMIT" clause in Postgres' SELECT command if necessary.)
(b)
Write five data modification commands on your PDA database. Most of these commands should be ``interesting,'' in the sense that they involve some complex feature, such as inserting the result of a query, updating several tuples at once, or deleting a set of tuples that is more than one but less than all the tuples in a relation. As with the queries in (1), you might want to try out your commands on small data before trying it on your full database. Hand in a script that shows your modification commands running in a convincing fashion.

(c)
Create two views on top of your database schema. Show your CREATE VIEW statements and the response of the system. Also, show a query involving each view and the system response (but truncate the response if there are more than a few tuples produced). Finally, show a script of what happens when you try to modify your view, say by inserting a new tuple into it. In theory, should either of your views be updatable? Tell why or why not? (Updatable views are discussed in Section 5.8.4 of the text. Essentially, a view is updatable if it is a selection on one base table.  But note, that PostgreSQL's implementation of views may not be updateable.)

(d)
In part (1) you probably discovered that some queries run very slowly over your large database. An important technique for improving the performance of queries is to create indexes. An index on an attribute A of relation R allows the database to find quickly all tuples in R with a given value for attribute A. This index is useful if a value of A is specified by your query (in the where-clause). It may also be useful if A is involved in a join that equates it to some other attribute. For example, in the query
     SELECT Dealers.address
FROM Drivers, Dealers
WHERE Drivers.name = 'joe'
AND Drivers.frequents = Dealers.name;
we might use an index on Drivers.name to help us find the tuple for driver Joe quickly. We might also like an index on Dealers.name, so we can take all the dealers Joe shops at and quickly find the tuples for those dealers to read their addresses.

In Postgres, you can get an index by the command:

     CREATE INDEX <IndexName> ON <RelName>(<Attribute List>);

Note:

If the attribute list contains more than one attribute, then the index requires values for all the listed attributes to find a tuple. That situation might be helpful if the attributes together form a key, for example. An illustration of the CREATE INDEX command is

     CREATE INDEX DriverInd ON Drinvers(name);
CREATE INDEX DealerInd ON Dealers(name);
which creates the two indexes mentioned above. To get rid of an index, you can say DROP INDEX followed by the name of the index. Notice that each index must have a name, even though we only refer to the name if we want to drop the index.

Create at least two useful indexes for your PDA. Run your queries from part (1) on your large database with the indexes and without the indexes. To time your commands, you may preface your query with EXPLAIN ANALYZE, such as:

        EXPLAIN ANALYZE <query>;

WARNING: "EXPLAIN ANALYZE" will actually run your query in order to give you the timing information.  That is fine for a generic SELECT-FROM-WHERE query, but could upset things if your query involves DELETE, INSERT, or UPDATE.  If that is the case, you may want to use the following approach:

BEGIN; EXPLAIN ANALYZE <query>;
ROLLBACK;

Naturally these times may be affected by external factors such as system load, etc. Still, you should see a dramatic difference between the execution times with indexes and the times without. Turn in a trace showing your commands to create indexes, and showing the relative times of query execution with and without indexes.

Note: Often, students discover that indexes appear to slow down the execution of queries. There are two issues you should consider:

    1. A query involving several relations will not speed up unless indexes to support all of the selections and joins are available. A single index may only increase the number of disk I/O's needed (to get index blocks), without affecting the query as a whole.
    2. The second time you run a query may take much less time than the first, because the second time data is cached in main memory. Data may be cached in the main memory of the database server. If many students are using Postgres at the same time, the machine's cache will probably drop your data if you wait a few seconds. We suggest that you perform only one timing experiment in a session; at least exit from psql and start it again. If you are getting strange results, you may have to wait several minutes between runs.

Acknowledgements:
  Most of the material for the project descriptions and assignments has been taken from Professor Ullman's website.