Data, Information and What a Database Really Is

Foundations of Database Systems · 20 min

Core CS · DBMS

Numbers do not mean anything by themselves

Data is what you recorded. Information is what someone can act on. Everything a DBMS does lives in the gap between them.

Turn five raw marks into an answer
Data · 78 65 35 82 90
Same numbers · mean 70, 80% passed

01 The idea

The gap between a value and a fact

Here are five numbers: 78, 65, 35, 82, 90. You cannot do anything with them. Not because they are wrong or incomplete, but because nothing has said what they count. They could be marks out of 100. They could be ages, or rupees, or the last five readings from a sensor. The numbers do not care and they will not tell you.

Now add one line: these are the marks five students scored in a DBMS mid-term, out of 100, and 40 is the pass mark. Not one digit changed. What changed is that questions became answerable. The class averaged 70. Nikhil topped with 90. Four of the five cleared the pass mark, so 80 per cent of the class passed, and Meera at 35 is the one who did not. The line you added is metadata, and everything after it is arithmetic.

This is also why “a database is a collection of related data” is the answer that gets a follow-up question. A folder of five spreadsheets is a collection of related data. What makes something a database is that the description of the data is stored inside it, next to the data, in a form the software itself can read. Ask a database what its columns are called and it answers. Ask a spreadsheet and you have to go and find the person who made it.

Data is what you recorded. Information is data that has been given context and processed to answer somebody’s question. The bytes on disk are identical in both cases.
DataRecorded values with nothing attached that says what they measure. 35 is data. So is Meera. Data is whatever survives if you delete every column heading.
InformationData that has been given context and processed so that it answers a question somebody actually asked. “Meera is the only student below the pass mark” is information.
MetadataData that describes data: a column’s name, its type, its width, whether it may be empty, and the rule its values must obey. It is what turns a byte into the number 78.

02 Worked example

One question, and the five layers under it

You type one line: SELECT student, marks FROM marks WHERE marks < 40;. Nothing in it names a file, a disk block or a byte offset. Follow the same five values from section 01 on their way to becoming an answer.

stored7865358290data · five integers, nothing attached
answer7865358290information · the only mark below the pass mark of 40

Those two rows hold the same bytes. Everything that happened between them happened in five layers, and only the first and the last are visible to you.

Your queryone line of SQL: a table and a condition, nothing else
DBMSparses it, checks you are allowed to read MARKS, plans the fetch
Data dictionarythe DBMS’s own tables: marks is a 2-byte integer, second in the row
Stored blocks8 KB of bytes on disk, none of them labelled
ResultMeera, 35 — one row you can act on

The highlighted node is the one students skip. Without the data dictionary the storage engine has a block of bytes and no way to cut it into fields, and the DBMS has a table name it cannot resolve. Metadata is not documentation sitting beside the database. It is a working part of every single read.

Take the middle three nodes away and you are back to a file plus a person who has to know its layout. That is how it was done before databases: the byte offsets lived inside every program that read the file, so the storage and the programs could never change independently. Moving that knowledge into the dictionary is what breaks the coupling, and it has a name. Physical data independence is the ability to change how data is stored without rewriting the code that reads it.

03 Mechanics

What the layer buys you, row by row

The interview version of this section is “file system versus DBMS”. Answer it as a list of properties, not a list of adjectives. Each row below is one thing that goes wrong with plain files and the named mechanism that fixes it.

What you are relying onA folder of filesA DBMS
Where the layout is written downinside every program that reads itonce, in the data dictionary
Two people writing at the same timelast writer winstransactions and locking
A crash halfway through a changea half-written fileatomicity: all of it or none of it
Stopping a bad value getting inwhatever the writer remembered to checktypes, NOT NULL, CHECK, foreign keys
Asking a question nobody planned forwrite a programwrite a query
Finding one row among a millionread the whole filean index the planner chooses
Who may see which partthe whole file, per operating-system userGRANT, per table and per role

The first row is the one everything else rests on, so it is worth opening up. Here is what the dictionary records about a single column, marks.marks, and what stops working if that entry is missing.

What is recordedFor marks.marksWhat breaks without it
Column namemarksYou address the value by byte offset instead, in every program that reads it.
Type and widthSMALLINT, 2 bytesThe byte 4E is the number 78 and also the letter N. Nothing decides between them.
Position in the rowsecondThe storage engine has a block of bytes and no way to cut it into fields.
NullabilityNOT NULL“Has not sat the exam yet” and “scored zero” become the same row.
ConstraintCHECK (marks BETWEEN 0 AND 100)A typo of 350 is stored, and every average computed after it is wrong.
Privilegesteacher may UPDATE, student may SELECTAnyone who can open the file can edit any mark in it.
Every one of those entries is written by the DBMS itself, when you run CREATE TABLE or ALTER TABLE, and read back with an ordinary SELECT. That is why the description can never drift away from the data it describes. A comment at the top of a spreadsheet can, and usually has.

05 Cheat sheet

Say this, not that

Every term here has a short answer that is correct and a shorter one that is correct but hollow. The hollow one is not wrong, which is exactly why it invites the follow-up question in the third column.

TermSay thisThe answer that invites a follow-up
Datarecorded values, no question attached“numbers” — then a name or a date is not data, which you do not want to defend.
Informationdata given context and processed to answer a question“processed data” — true and empty. The next question is what processing, and context does half the work with none.
Databasean integrated, shared, self-describing store of data and its metadata“a collection of related data” — so is a folder of spreadsheets, and that is the follow-up.
DBMSthe software that owns the files and answers queries about them“software to manage a database” — circular. Name what it does: dictionary, constraints, transactions, recovery, access control.
Metadatanames, types, widths, nullability, constraints, owners, indexes“data about data” — the follow-up is “such as?” and the list above is the whole answer.
Data dictionarythe DBMS’s own tables holding all of that metadata“where the schema is kept” — vague. Name a real one you have queried.
Database systemthe database, plus the DBMS, plus the applications and users on top“same as a database” — then you cannot say which layer enforces a constraint.
The self-describing testOpen a CSV and somebody has to tell you what column three is. Open a database and you can ask it, because the description is stored in the same system as the data, in tables you query the same way. That single property is most of the difference between a file and a database.
The labels swapA delivery address is information to the driver and data to the job that counts orders per pincode. Nothing about the value changed; the question did. Any definition that fixes a value permanently as one or the other is wrong.
The layer is not freeDictionary lookups, privilege checks and transaction bookkeeping are work that a plain file read does not do. For one program reading one flat file start to finish, raw file I/O wins. You pay the overhead for the guarantees, not for speed.

06 Where & why

Where the metadata actually lives

The data dictionary is not an abstraction from a textbook. It is a set of tables with names, in every database you have ever opened, and you can read them today. An answer that names one lands very differently from one that says “the metadata is stored somewhere in the system”.

PostgreSQL · MySQL
The dictionary is queried with ordinary SQL

PostgreSQL keeps its catalogs in the pg_catalog schema and exposes the standard view of them as information_schema. MySQL exposes information_schema too, and SHOW COLUMNS FROM marks is the shortcut. Same SELECT, same result grid, same permissions model as the data itself.

Oracle
Where the phrase comes from

Oracle calls it the data dictionary outright: base tables owned by SYS, read through views such as USER_TAB_COLUMNS and ALL_TABLES. The USER_, ALL_ and DBA_ prefixes are themselves an access-control decision expressed in metadata.

SQLite
One file, and it still describes itself

sqlite_master holds one row per table, index, view and trigger, and its sql column stores the original CREATE statement text. The whole database, data and dictionary together, is a single file on disk. Self-describing does not require a server.

Excel · CSV
The case where the metadata lives in someone’s head

A column headed marks holding the value 35 has a heading and nothing else: no type, no constraint, no owner. The rule that marks run 0 to 100 exists only as the habit of whoever maintains the sheet, so nothing stops a typo of 350 and nothing records who typed it. Every argument for a DBMS starts here.

If you can name where the metadata lives in one database you have actually used, and say what you found there, the “what is a DBMS” question is over in a sentence. If you cannot, every answer you give is a definition somebody else wrote.

07 Interview questions

What they actually ask

This is the first question in most DBMS interviews and the one candidates most often answer from memory. The give-away is a definition that cannot survive one follow-up. Answer each of these in about twenty seconds, out loud.

What is the difference between data and information?
Data is what was recorded; information is data placed in context and processed so that it answers a question. 78, 65, 35, 82, 90 is data. “The DBMS mid-term averaged 70 and 80 per cent of the class passed” is information, and it is the same five numbers. The distinction is about the question being asked, not about the values.
Give me something that is information to one system and data to another.
A delivery address is information to the driver, who uses it to reach a door. The same address is data to the analytics job that groups last month’s orders by pincode. Nothing about the value changed; the question changed. Interviewers ask this to check you can apply the distinction rather than recite “processed data”.
Why is “data is raw, information is processed” an incomplete answer?
Because it leaves out context, which does most of the work. Attaching the column name marks and a student to each number processes nothing at all, and the five values already say far more than they did. Say both halves: context turns values into facts, and processing turns facts into answers.
What is a database?
A shared, integrated collection of related data together with the metadata that describes it, stored so that many users and programs can use it at once without any of them knowing the file layout. Stop at “a collection of related data” and the next question is how that differs from a folder of spreadsheets. The word that saves you is self-describing.
So how is it actually different from a folder of files?
A file hands you bytes and the layout lives in whichever program reads it. A database stores the layout beside the data, enforces constraints on every write, lets several people write at once without corrupting each other, and can undo a half-finished change after a crash. Those four are the answer: self-description, integrity, concurrency, recovery.
What is a DBMS, and what does it do while running one SELECT?
The DBMS is the software layer that owns the files. On one query it parses the statement, checks your privileges, looks up the table and column definitions in the data dictionary, plans how to fetch the rows, reads blocks off disk and cuts them into fields using the offsets the dictionary gave it, then returns the result. Your query never named a file or a byte offset, and that is the point.
What is metadata? Name five kinds.
Data that describes data. Column name, data type, width, whether nulls are allowed, and the constraint the values must satisfy; also the primary key, indexes, owner and grants. Without the type, the byte 4E is equally the number 78 and the letter N. Metadata is what makes a byte a value.
What is the data dictionary, and where is it in a database you have used?
It is the DBMS’s own set of tables holding all the metadata. In PostgreSQL it is pg_catalog, exposed through information_schema. In Oracle it is the dictionary owned by SYS, read through views like USER_TAB_COLUMNS. In SQLite it is the single sqlite_master table. You read all of them with the same SELECT you use on ordinary data.
Who writes to the data dictionary?
The DBMS does, not you. CREATE TABLE and ALTER TABLE are the statements that change it; you read the catalog but you do not hand-edit it. That is why the description can never fall out of step with the data it describes, and it is the difference between metadata inside a database and a comment at the top of a CSV.
Database, DBMS, database system: are they the same thing?
No. The database is the stored data plus its metadata. The DBMS is the software that manages it. The database system is both of those plus the applications and the users on top. Interviewers ask because candidates use “database” for all three, and it stops being a word game the moment you are asked which layer enforces a constraint.
When is a plain file the right answer instead of a database?
When one program owns the data, reads it start to finish, and nothing else writes it. Application logs, a config file, a model checkpoint. You would be paying for concurrency control, recovery and a dictionary you never query. The trade flips the moment a second writer or a second kind of question shows up.

08 Practice problems

Six to work through

None of these has a single-word answer. For each one, write down what you know, what you assumed, and which of the two came out of the data rather than out of your head.

Label the lines

Easy
For each line say whether it is data, metadata or information, and where it is information, name the question it answers: (a) 1042, (b) order_id INTEGER NOT NULL, (c) Order 1042 shipped on 14 March, (d) the orders table has three indexes, (e) average order value in March was 1,180.
Follow-up
Two of the five change category depending on who you say is asking, and one of them describes the table rather than describing any order. A list of five single labels is not the full answer.
Show the hint
For each line, settle what it describes before you argue about what it is worth.

Two more questions

Easy
Using only the five marks in this lesson and the metadata attached to them, write two questions the table can answer and one it cannot, and for the one it cannot, name the single extra column that would fix it.
Follow-up
The question it cannot answer has to fail for want of context, not for want of arithmetic. “What is the median?” does not count, because the table already holds everything that question needs.
Show the hint
Think of something a teacher would want to know about these five students that no amount of arithmetic on the marks column can produce.

Where does the rule live?

Medium
Two teams keep the same customer list. Team A keeps it in a CSV that two of their scripts read and write; Team B keeps it in a PostgreSQL table with a unique constraint on email. Describe the exact sequence of events that puts a duplicate email into Team A’s list, and say why the same sequence cannot do it to Team B.
Follow-up
Both of Team A’s scripts check for an existing email before writing, and both checks are written correctly. The duplicate still gets in, so the answer is about timing rather than about a missing check.
Show the hint
Put the two scripts on one timeline and look hard at the gap between the moment each one checks and the moment each one writes.

Read the dictionary first

Medium
You are handed a database you have never seen, no documentation, and no access to the application source. Write the ordered list of questions you would answer from the catalog alone before running a single SELECT against real data, and say what each answer unblocks.
Follow-up
The obvious first move is not available to you: you cannot select from a table until you know it is there. At least one question on a good list has nothing to do with columns, and leaving it out is how people misread a schema.
Show the hint
Order the questions by what each one makes possible next, not by what you are curious about.

Same numbers, new rule

Medium
The five marks are reused for a scholarship whose rule is “the top twenty per cent of the class”. State who qualifies, list every piece of context you had to bring in that is not in the five rows, and say where you would store it so that next term’s report cannot quietly disagree with this one.
Follow-up
The arithmetic takes seconds. The interesting part is counting the things you used that came out of your head rather than out of the table, and one of them only becomes visible if two students tie.
Show the hint
Write down every input you used and tick the ones a query could have fetched for you.

Widen one column

Hard
A legacy system stores each order as a fixed-width binary record: bytes 0 to 3 the order id, 4 to 33 the customer name, 34 to 37 the amount in paise, byte 38 a status code. Six programs read this file. Describe everything that must change to widen the customer name from 30 bytes to 50, then describe what changes in a database holding the same orders in a table, and name the property that accounts for the difference.
Follow-up
The database answer is not “nothing changes”. One thing does, and being honest about which one is what separates a real answer from a slogan.
Show the hint
Find every place the number 30 is written down in each design before you write a word about the migration.