Volatile vs non-volatile storage
Volatile loses its contents when power is switched off (cache, main memory). Non-volatile keeps them (secondary and tertiary storage, battery-backed main memory).
Flash memory: reads, writes, power failure
Data survives power failure; reads are roughly as fast as main memory; writes are slow (few microseconds) and erase is slower.
Magnetic disk: role and access type
The primary medium for long-term storage (typically the entire database); direct-access — data can be read in any order, unlike tape.
Tape storage: use and access
Used primarily for backup and archival data; sequential-access — much slower than disk; very high capacity.
The storage hierarchy from fastest to slowest
Cache, main memory, flash memory, magnetic disk, optical disk, magnetic tapes.
Primary, secondary and tertiary storage
Primary: fastest, volatile (cache, main memory). Secondary (on-line): non-volatile, moderately fast (flash, magnetic disks). Tertiary (off-line): non-volatile, slow (tape, optical).
Sector and cylinder
Sector: a division of a track, the smallest unit of data that can be read or written (typically 512 bytes). Cylinder i: the ith track of all the platters.
Three things the disk controller does to protect data
Computes and attaches checksums to each sector; reads back a sector after writing it; remaps bad sectors.
SAN vs NAS
SAN: many disks connected by a high-speed network to servers. NAS: networked storage with a file system interface (networked file system protocol), not a disk interface.
Access time = ?
Seek time + rotational latency — the time from issuing a request until data transfer begins.
Average seek time and average rotational latency
Each is 1/2 of its worst case (average seek would be 1/3 if all tracks had the same number of sectors).
MTTF of 1,200,000 hours for 1000 new disks means…
On average, one disk will fail every 1200 hours. MTTF decreases as the disk ages.
Block size trade-off
Smaller blocks: more transfers from disk. Larger blocks: more space wasted due to partially filled blocks (typical 4–16 KB).
Why is writing to a log disk very fast?
It writes a sequential log of block updates, so no seeks are required.
RAID: what does it stand for, and what does it provide?
Redundant Arrays of Independent (originally Inexpensive) Disks — a view of a single disk of high capacity and speed (parallel disks) and high reliability (redundant data).
100 disks, each MTTF 100,000 hours: system MTTF?
1000 hours (about 41 days) — some disk out of many fails much sooner than one specific disk.
Mirroring (shadowing)
Duplicate every disk: every write goes to both disks, reads from either; data is lost only if a disk and its mirror both fail before repair.
Block-level striping with n disks
Block i of a file goes to disk (i mod n) + 1.
RAID 0 and RAID 1
RAID 0: block striping, non-redundant (data loss not critical). RAID 1: mirrored disks with block striping — best write performance; popular for log files.
RAID 4: scheme and weakness
Block-interleaved parity with the parity block on a separate disk; the parity disk is a bottleneck for independent block writes.
RAID 5 vs RAID 4
RAID 5 distributes data and parity among all N + 1 disks, giving higher I/O rates and avoiding the parity-disk bottleneck; it subsumes level 4.
RAID 6
P+Q redundancy — like level 5 with extra redundant information to guard against multiple disk failures; better reliability at higher cost.
RAID 1 or RAID 5: which for what?
Level 1: high update environments such as log disks (better write performance). Level 5: low update rate and large amounts of data.
Hot swapping and data scrubbing
Hot swapping: replacing a disk while the system runs, without power down. Data scrubbing: continually scanning for latent failures and recovering from copy/parity.
Where does fixed-length record i start?
At byte n × (i − 1), where n is the size of each record.
Free list
The file header stores the address of the first deleted record; each deleted record stores the address of the next — pointers linking all free records.
Variable-length records: how are variable attributes and nulls stored?
Variable-length attributes as a fixed-size (offset, length), with data after the fixed-length attributes; nulls by a null-value bitmap.
Slotted page header and pointers
Header: number of record entries, end of free space, location and size of each record. Pointers point to the header entry, not the record, so records can move.
Heap vs sequential vs hashing file organization
Heap: records anywhere, no ordering. Sequential: ordered by the search key. Hashing: a hash function on an attribute gives the block.
Sequential file: inserting when there is no free space
Insert the record in an overflow block and update the pointer chain; reorganize the file from time to time.
Multitable clustering of department and instructor: good and bad for
Good for department ⋈ instructor and one department with its instructors; bad for queries involving only department (and gives variable size records).
What does the data dictionary (system catalog) store?
Metadata: relations and their attributes (names, types, lengths), views, integrity constraints, user and accounting information including passwords, statistics (tuples per relation), physical organization and location, indices.
When is a replaced buffer block written back to disk?
Only if it was modified since the most recent time it was written to/fetched from the disk.
Why can LRU be a bad buffer-replacement strategy?
For access patterns with repeated scans of data, e.g., a nested-loops join; a mixed strategy with hints from the query optimizer is preferable.
Pinned block, toss-immediate, MRU
Pinned: not allowed to be written back to disk. Toss-immediate: free the block once its final tuple is processed. MRU: pin the block in use; after its final tuple it is unpinned and becomes the most recently used block.