5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 1/47

Business Fax Solutions Send & Receive Faxes Online. HIPAA, SOX & GLB Compliant.

Tutorial: Step by Step Database Design in SQLFeb 22, 2015 23,195 views 67 Likes 7 Comments

Please check out my related article "How did the modern relational database

come to be?" which is currently trending in Big Data and follow me for daily

articles on technology, digital marketing, psychology and pharmaceuticals.

Database Design and Implementation is applicable for whatever industryyour in. Here is a step by step approach to designing and implementing adatabase in your organisation, using specific data from a sweet shopcase study I implemented during my M.Sc. in Software & InformationSystems. By the end of this tutorial, you will know about databases,advantages of databases system over regular file system, the steps of adatabase design process, software development lifecycle, qualities of awell built database, relations and relationships, data integrity, and more.Databases are used in every industry, including the pharmaceuticalindustry.Background of Databases

The database system approach to data management overcomes many of the

shortcomings of the oldfashioned file system approach. One of the key features

of a database system is that data is stored as a single logical unit. What this means

is that although the data may be spread across multiple physical files, the

David McCaldinMSc Student / Penetration Tester / Medical Devices / CSV Follow

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 2/47

database conveys the data as being located in a single data repository. Organizing

data in single logical repository allows for easy manipulation and querying of the

data, in contrast with traditional file systems where the programmer must specify

what and how the data retrieval is done.

With database systems, it need only be specified what must be done, the DBMS

(Database Management System) does the rest. Another advantage of the database

approach is that, because data is located in one single database, data in different

physical locations need not be duplicated. The database software can interact

with all the data in the database. Non duplication of data is one way of

maintaining the integrity of the data. When data is allowed to be duplicated,

errors can happen if one instance of the data is altered and another instance

remains the same. When data is allowed to be duplicated, more maintenance and

system resources are required to ensure that data is always integral.

One of the greatest benefits of databases is that data can be shared or secured

among users or applications. There is more control and accountability over how

the data is managed because the data all resides in one database.

If there are shortcomings to database systems, its that much more powerful and

sophisticated software is needed to control the database and designing the

software and database can be extremely time consuming. More extensive

knowledge of how to use the database is required, thus making the database

system less user friendly than traditional file systems. Since the database is one

logical repository, even a small error can damage the entire database and reduce

the integrity of the data. One benchmark of a good database is one which is

complete, integral, simple, understandable, flexible and implementable. Batini et

al says that database modelling strives for a “non redundant, unified

representation of all data managed in an organization”. By following the database

software development lifecycle methodology, and by using the data models, the

database design ideals are fulfilled and will minimize the disadvantages.

Databases & the Software Development Lifecycle

The steps in developing any application can be represented as a linear sequence

where each step in the sequence is a function, which passes its output to its

successor function. Adherence of a ‘waterfall development model’ ensures quality

software, which is ‘complete’, ‘efficient’, ‘usable’, ‘consistent’, ‘correct’ and

‘flexible’. These traits are also some of the core underpinnings of a wellbuilt

database. The waterfall model can be applied to database design theory as

effectively as it is applied to other software engineering theory. The steps can be

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 3/47

summarized as follows:

Requirements specification > Analysis > Conceptual design >Implementation Design> Physical Schema Design and Optimisation

In consultation with all potential users of the database, a database designer’s first

step is to draw up a data requirements document. The requirements document

contains a concise and nontechnical summary of what data items will be stored

in the database, and how the various data items relate to one another. Taking the

‘data requirements document’, further analysis is done to give meaning to the

data items, e.g. define the more detailed attributes of the data and define

constraints if needed. The result of this analysis is a ‘preliminary specifications’

document. Taking the specifications document, the database designer models

how the information is viewed by the database system and is how it is processed

and conveyed to the end user. In the implementation design phase, the

conceptual design is translated into a more lowlevel, DBMS specific design.

Data Models & Schemas as a Means of Capturing Data

The database development design phases brings up the concept of ‘data models’.

Data models are diagrams or schemas, which are used to present the data

requirements at different levels of abstraction. The first step in the Database

Development Life Cycle is to draw up a requirements document.

Figure 1: A basic example of a requirements document

The requirements document can then be analysed and turned into a basic data set

(as shown in Figure 2) which can be converted into a conceptual model. The end

result of the conceptual design phase is a conceptual data model (Figure 3), which

provides little information of how the database system will eventually be

implemented. The conceptual data model is simply a highlevel overview of the

database system.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 4/47

Figure 2: A Database Data Set is the Result of analyzing the Information from

the Requirements Phase. The Primary Keys are Underlined.

Figure 3: A Normalized EntityRelationship model (ERD) in Crow’s Foot

Notation is an Example of a Conceptual Data Model and provides no

information of how the database system will eventually be implemented

In the implementation design phase, the conceptual data model is translated into

a ‘logical’ representation of the database system. The logical data model conveys

the “logical functioning and structure” of the database and describes ‘how the

data is stored’ (e.g. what tables are used, what constraints are applied) but is not

specific to any DBMS. Logical database model is a lowerlevel conceptual model,

which must be translated to a physical design.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 5/47

Figure 4: In the implementation design phase, the conceptual data model (ERD)

is translated into a ‘logical’ representation (logical schema) of the database

system: a data dictionary.

Physical modelling deals with the “representational aspects” and the “operational

aspects” of the database, i.e. the internal DBMS specific operations and processes

and how the DBMS interacts with the data, the database and the user. The

translation from logical design to physical design assigns “functions” to both the

machine (the DBMS) and to the user, functions such as storage and security, and

additional aspects such as consistency (of data) and learnability are dealt with in

the physical model/schema. Practically speaking, a physical schema is the SQL

code used to build the database.

One benchmark of a good database is one, which is complete, integral, simple,

understandable, flexible and implementable. Database modelling strives for a

nonredundant, unified representation of all data managed in an organization. By

following the above methodology, and by using the data models, these database

design ideals are fulfilled. In conclusion, here are two examples of why using data

models is paramount to capturing and conveying data requirements of the

information system:

1. By drawing up a ‘logical model’, extra data items can be added more easily in

that model then in the physical model. A database design that can change

easily according to needs of the company is important, because it ensures the

final database system is complete and uptodate.

2. Another consideration is understandability. By initially creating a ‘conceptual

model’, both the designer and the organization are able to understand the

database design and decide if it is complete or not. If there were no

‘conceptual model’, the organization would not be able to ‘conceptualize’ the

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 6/47

database design and make sure that it actually represents all the data

requirements of the organization.

3. By creating a physical model, the designers can have a low level overview of

how the database system would operate before it is actually implemented.

SQL Statements – Implementing the Database

The final step is to physically implement the logical design which was illustrated

in Figure 4. To physically implement the database, SQL can be used. These are

the main steps in implementing the database:

1. Create the Database Tables

The tables come directly from the information contained in the Data Dictionary.

The following blocks of code each represent a row in the data dictionary and are

executed one after another. The blocks of “create table” code contain the details of

all the data items (COMPANY, SUPPLIER, PURCHASES, EMPLOYEE etc), their

attributes (names, ages, costs, numbers and other details), the Relationships

between the data items, the Keys and Data Integrity Rules. All of this information

is already detailed in the Data Dictionary, but now we are actually converting it

and implementing it in a physical database system.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 7/47

Explanation:

1. The create table statement indicates that you want a table to be created.

2. The name of the table proceeds the first '('

3. The table attributes and Data Integrity Rules are defined within the two

parentheses. How the table relates to other tables is also defined within the

two parentheses (e.g. by defining FOREIGN KEYS)

4. The not null statement means that if you try to populate the table with

values, but leave the value of that attribute empty, you will get an error.

5. The varchar2 (19) means a string of 19 characters.

6. The number (6,2) means a number which can have a number of up to 6

digits, 2 of them being after the decimal place, e.g. you can have a number

from 0.0 to 1234.56.

7. The date means that that attribute will be represented as a date within the

Tutorial: Step by Step DatabaseDesign in SQLDavid McCaldin

Google dodges a $9 billionbullet; The Ashley Madisonfiles: adulterers are bad boys …business, too; and more newsJohn C Abell

Facebook and others thinkmessaging as a platform is thenext big thing. Are they right?Chris Moore

Pulse

What is LinkedIn? Join Today Sign In

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 8/47

database system.

8. The CONSTRAINT statement means that a constraint is being defined. This

statement will be used to describe which of the attributes are primary keys

and which (if any) of the attributes are foreign keys (referencing another

table).

9. The CONSTRAINT statement is in the form CONSTRAINT xxx PRIMARY

KEY(name_of_attribute_that_you_want_as_the_primary_key) or

CONSTRAINT yyy FOREIGN

KEY(name_of_attribute_that_references_another_table) REFERENCES

hhh(name_of_attribute_that_references_another_table) where the values

of xxx and yyy are just arbitrarily made up names which are not important.

hhh is the name of the base table being referenced.

2. Populate the tables

Use SQL statements to populate each table with specific data (such as employee

names, ages, wages etc).

3. Query the database.

Write SQL statements to obtain information and knowledge about the company,

e.g. how many employees are there, total profit etc.

Keys & Data Integrity Rules

Data integrity rules are a core component of a data model. Integrity rules

“implicitly or explicitly define the set of consistent database state(s)”. So, integrity

rules “ensure that database states and changes of state confirm to specified rules.”

Data integrity rules are of two types: Entity integrity rules and Referential

integrity rules..

How do keys relate to ensuring that changes in database states confirm to

specified rules?

Well, for example, you could ensure that the primary key of an entity cannot be

null. This is one way of ensuring entity integrity. If primary keys were allowed to

be null, then there would be no way of ensuring that individual entities were

uniquely identifiable. If you cannot ensure that individual entities are uniquely

identifiable then you can’t ensure that the database is ‘integral’, which is a core

property of a properly designed data model. So, by ensuring that keys follow

certain rules, you can ensure integrity of data.

How Technology HijacksPeople’s Minds — from aMagician and Google’s Desig…EthicistTristan Harris

CEO pay isn't slowing;Investors can't get enough ofnow$18 billion Snapchat, an…more newsIsabelle Roughol

Hiring Managers: Stop Insultingthe Job CandidatesBrian de Haaff

Here’s Why You’re Wrong ToThrow Shade At The TSA AndTheir Fired Executive

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 9/47

Written byDavid McCaldin

7 comments

Another way of enforcing integrity of data via keys, is to ensure that, if two tables

are related to each other, an attribute of one relation must be the same as the

primary attribute (primary key) of the other one. Enforcing this rule ensures

referential integrity of data.

So, we do need integrity rules, and proper defining of keys are a means of

enforcing them.

Relationships

When initially explaining the ‘relational model’, E.F. Codd proposed that users

should be abstracted from the internal representation of the data, such that if the

internal representation of that data were to change (e.g. because of system

growth), the way that the user perceives the data should remain unchanged. This

is why he proposed that users should only interact with a “collection of time

varying relationships” i.e. a user should only know the “name of the relationship

together with its domain” (e.g. department is the domain of employees, and the

employees are ‘owned’ by the department) rather than the relation (table) itself.

In using the terms ‘relation’ and ‘table’ as synonyms, Codd must have implied

that a table should be viewed in terms of its ‘relationship’ with other tables.

Relationships are what bind the relations/tables in a database together, so proper

understanding is needed.

Incorrect understanding of relationships may lead to incorrectly defined

relationships between tables. Incorrectly defined relationships between tables

could lead to data not being updated correctly in some tables, or could cause a

data item to be unnecessarily duplicated in another table. All this may lead to

incomplete ‘information’ in the database, which in turn results in incomplete

‘knowledge’.

Featured In Big Data

Databases, Software Development, Data Analysis

Follow

Like Comment 67 likes

Add your comment

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 10/47

Google dodges a $9 billion bullet; The AshleyMadison files: adulterers are bad boys in business,too; and more newsMay 26, 2016 59,720 views 494 Likes 17 Comments

Facebook and Microsoft are building what will be "the highest

capacity subsea cable to ever cross the Atlantic." There are hundreds, for

Huda Naiem Mohamed OsmanInterpreter: Language Resources, Tax Returns' Preparer,Programmer/Developer of Database/System

David McCaldin :) I am speechless towards your conise course onDatabases. You have managed to narrate the knowledge and conceptsfluently and captively. You have excelled in your choice of models todemonstrate how can one make an efficient database file. Output display isessential and will follow naturally, once the foundation is set correctly. Allrelevant program codes and queries will need learning, but all these willlead you nowhere if you do not know how to initiate your data system andwarehouse TO WORK ON with the latter coding, queries and reporting.

Thank you on behalf of my peer readers and on behalf of myself for thetreasured input.

Like(3) Reply February 24, 2015

Hamza Tariq, David McCaldin, and OLAYEMI OYEWALE

Show More

John C AbellManaging Editor, News at LinkedIn Ex Reuters Ex Wired Follow

Popular

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 11/47

the record, but demand keeps growing. This one should be done by October 2017,

CNBC reports. Data infrastructure buildout is now the purview of suppliers, not

telecommunication intermediaries; two years ago Alphabet announced a Pacific

cable project with five Asiabased telecom companies.

So Facebook and Microsoft are going to build a giant underseacable across the Atlantic—oh, and it's 160TB/sec bit.ly/249ApCV1:08 PM 26 May 2016

262 176

WIRED @WIRED

Facebook and Microsoft Are Laying a Giant Cable Across th…Internet giants are starting to build enormous networks of their own,taking over the role traditionally played by telecom companies.wired.com

Follow

More evidence the 2017 iPhone will have a (mostly) glass case. The

chairman of "long time iPhone chassis maker Catcher Technology" told his

annual meeting it's a sure thing, reports Forbes. It would be reinforced with a

steel case, which we think could double (uh oh) as an antenna.

Google prevailed in a $9 billion lawsuit brought Oracle alleging the

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 12/47

Written byJohn C Abell

17 comments

search giant had violated its Java copyright to develop Android. This

case has bounced around various courts for years (the Supreme Court declined to

hear the federal one), and Oracle said it would appeal the verdict, by a California

state jury. For the nonelementary version, read Greg Leffler's analysis.

Make of this what you will: A relatively small number of Ashley Madison

members registered with work addresses. But it was enough for researchers to

draw some conclusions about their companies. On the downside: They were were

likely to have been disciplined by the SEC, and were above average in "bribery

and fraud scandals, tax disputes, human rights violations and product quality

problems," observes Financial Times columnist Michael Skapinker. On the

upside: These establishments "also appeared to be more inventive and creative"

— above average in patents, which "covered a wider range of technologies," and

"tended to be in highgrowth sectors."

Donald Trump clinched the Republican presidential nomination with

1,239 delegates, according to a count first reported by the AP.

Cover Art: Illustrations of herostyle G7 leaders built for a NGO to encourage

them to be Universal Health Coverage superheroes, Ise city, Mie prefecture on

May 26, 2016. World leaders kick off two days of G7 talks in Japan on May 26

with the creaky global economy, terrorism, refugees, China's controversial

maritime claims, and a possible Brexit headlining their packed agenda.

Featured In Technology, Daily Digest

Follow

Like Comment 497 likes

Andy BouraSenior Information Security Architect Thomson Reuters (Director SecureProduct Architecture | CISSP | MPhys )

Justice in Google vs Oracle. I read the judges notes from the previous trial itwas pretty clear from that Oracle were having a punt to see if they could getanything. It's really important precident for software engineering based onmy understanding APIs are not copyrightable.

Add your comment

Popular

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 13/47

Facebook and others think messaging as a platformis the next big thing. Are they right?May 26, 2016 10,461 views 236 Likes 26 Comments

Like(5) Reply(4) 17 hours ago

Suzy Ekman, Diego Valero Seyffert, Beñat Galdós Aizpurua, +2

Andy BouraSenior Information Security Architect Thomson Reuters(Director Secure Product Architecture | CISSP | MPhys )

To be clear. Google didn't develope an application in Java that would have been fine. They created a whole new virtualmachine to run Android Java applications without payingOracle for their virtual machine.

Like 5 hours ago

Khaled HussainSenior Systems Engineer at GlobalCharge

Erik Grob I think they'd absolutely want you to develop yoursoftware in Java on the premise that if your software turns outto be successful, they'd take "a punt to see if they could getanything" from you, as Andy Boura speculates. Perhaps nowthey won't...

Like 5 hours ago

Show More

Show More

Chris MooreFollow

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 14/47

As investors in the mobile ecosystem, we are constantly working to better

understand and predict how the unique aspects of the mobile computing platform

will allow entrepreneurs to build new and lasting businesses across industries.

Inspired by recent industry developments and conversations we’ve been having

with friends and founders creating mobilefirst services, we decided to open

source more of our conversations and create a forum to surface different

perspectives on mobile related issues that have relevance across a range of

categories.

Our intent is to share opinions and practical advice around what’s working as well

as what’s not. We’d love for more folks to join the discussion as our hope is that

this process will accelerate our collective knowledge. Look for a mix of industry

commentary, metrics and analysis, guest posts and quick takes in this vein (like

this perspective on Gboard from my partner Jamie Davidson).

Lately we’ve been awash in the messagingasaplatform discussions as all the

major players are coming out with messaging + AI plays. Last week Google

announced Allo as a kind of new smart messaging app. Facebook’s F8 messaging

news had already been top of mind as both a source of promise and a painful

reminder of how challenging the current app discovery and distribution structure

really is. Could messaging bots become a new distribution channel for app

providers? The people we’ve surveyed are intrigued but skeptical. Intrigued for

the obvious reason that everyone is looking for another way to reach consumers

where they’re spending time. But skeptical based on the applicability of the chat

based interface for the full range of apps, and the inevitability of the same

discovery challenges if the model gains awareness and traction.

One app whose approach to distribution and engagement could inform us more

broadly is Riffsy (also a Redpoint portfolio company). At the recent Mobile Apps

Unlocked conference (an awesome event organized by Grow.co founders Adam

Lovallo and Jay Weintraub) I did an onstage interview with Rffsy founder and

CEO David McIntosh where we talked about Riffsy’s approach to leveraging

messaging as a platform. David told the congregation of mobile app thinkers that

making Riffsy available across multiple messaging platforms was key to achieving

scale and encouraging repeat users. Riffsy works well on iOS, Kik, Twitter, and

Facebook Messenger among others. Its user base has shared tens of billions of

GIFs across multiple mobile messaging platforms no small feat given the

challenges of platform fragmentation. App makers here in the U.S. can’t borrow

from the Tencent playbook where the WeChat platform dominates the messaging

app landscape and serves as the de facto portal for 3rd party applications. But I’d

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 15/47

Written byChris Moore

26 comments

argue that makes the whole landscape more interesting – and an area we think is

still in its early stages here in the U.S.

Will messenger as a platform be a THING? And what does that portend for all the

platforms and app makers competing for our attention as well as the mobile

ecosystem at large? As we’re actively exploring other ideas to tackle the problem

of getting and keeping people’s attention in this crowded mobile landscape, we’re

taking a fresh look at the app as we know it. The time is right for new ideas about

messaging as a platform and how we can build an effective distribution and

engagement model for mobile content and services.

Featured In Technology, Entrepreneurship, Editor's Picks, VC & Private Equity, Mobile

Venture Capital, Mobile Applications, StartUps

Follow

Like Comment 236 likes

Mark ParrishSr. IT Project Manager, Product Development at TMG Health

Yawn! A fine example of a solution in search of a problem. Another intrusioninto being a true human that interacts with society as needed. As MontyPython says, this will increase " SPAM SPAM SPAM". Do businesses reallyneed to be attached to customers via messaging apps?

Like(3) Reply(3) 19 hours ago

Jonathan de Potter, Brian Hochstetter, and Robert W. Kapfhammer

Taitusi VuatakiProject Coordinator

Nope just Facebook

Like 5 hours ago

Sr. IT Project Manager, Product Development at TMG Health

It's very much missing the point as a solution... the problem,that is alluded to is that they want a mechanism that will getpeople to buy things via a chat system; the bottleneck with thatis the psychology of buying. Until this aspect is fullycharacterised, then Dallas is completely correct. For me thesolution is not the platform, but the mechanism of purchase. Aplatform for sales (or maybe "arena") could be a supermarket,but the mechanism is the till; influencing factors are the

Add your comment

Popular

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 16/47

How Technology Hijacks People’s Minds — from aMagician and Google’s Design EthicistMay 26, 2016 7,013 views 354 Likes 39 Comments

I’m an expert on how technology hijacks our psychological vulnerabilities. That’s

why I spent the last three years as a Design Ethicist at Google caring about how to

design things in a way that defends a billion people’s minds from getting hijacked.

When using technology, we often focus optimistically on all the things it does for

us. But I want you to show you where it might do the opposite.

ambience (music, temperature, aroma, colours), the absenceof clocks, the layout of the products, etc... you could argue thata solution mechanism might be having the till fitted to thetrolley you push around, or at the end of each aisle, etc... Thereal platform, might be the trolley or the manned till and selfservice till, and is very much a dependency of the mechanism.That's my tuppence. All this wideeyed IoT overethusiasm issuffering from a failure of problem analysis and requirementscapture.

Like(2) 7 hours ago

Kristian Wilde and Dallas Stephens

Show More

Show More

Tristan HarrisDesign Ethics & Product Philosopher at Google Follow

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 17/47

Where does technology exploit our minds’ weaknesses?

I learned to think this way when I was a magician. Magicians start by looking for

blind spots, edges, vulnerabilities and limits of people’s perception, so they can

influence what people do without them even realizing it. Once you know how to

push people’s buttons, you can play them like a piano.

(That’s me performing sleight of hand magic at my mother’s birthday party)

And this is exactly what product designers do to your mind. They play your

psychological vulnerabilities (consciously and unconsciously) against you in the

race to grab your attention.

I want to show you how they do it.

Hijack #1: If You Control the Menu, You Control the Choices

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 18/47

Western Culture is built around ideals of individual choice and freedom. Millions

of us fiercely defend our right to make “free” choices, while we ignore how those

choices are manipulated upstream by menus we didn’t choose in the first place.

This is exactly what magicians do. They give people the illusion of free choice

while architecting the menu so that they win, no matter what you choose. I can’t

emphasize enough how deep this insight is.

When people are given a menu of choices, they rarely ask:

“what’s not on the menu?”

“why am I being given these options and not others?”

“do I know the menu provider’s goals?”

“is this menu empowering for my original need, or are the choices actually a

distraction?” (e.g. an overwhelmingly array of toothpastes)

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 19/47

(How empowering is this menu of choices for the need, “I ran out of

toothpaste”?)

For example, imagine you’re out with friends on a Tuesday night and want to

keep the conversation going. You open Yelp to find nearby recommendations and

see a list of bars. The group turns into a huddle of faces staring down at their

phones comparing bars. They scrutinize the photos of each, comparing cocktail

drinks. Is this menu still relevant to the original desire of the group?

It’s not that bars aren’t a good choice, it’s that Yelp substituted the group’s

original question (“where can we go to keep talking?”) with a different question

(“what’s a bar with good photos of cocktails?”) all by shaping the menu.

Moreover, the group falls for the illusion that Yelp’s menu represents acomplete

set of choices for where to go. While looking down at their phones, they don’t see

the park across the street with a band playing live music. They miss the popup

gallery on the other side of the street serving crepes and coffee. Neither of those

show up on Yelp’s menu.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 20/47

(Yelp subtly reframes the group’s need “where can we go to keep talking?” in

terms of photos of cocktails served.)

The more choices technology gives us in nearly every domain of our lives

(information, events, places to go, friends, dating, jobs) — the more we assume

that our phone is always the most empowering and useful menu to pick from. Is

it?

The “most empowering” menu is different than the menu that has the

most choices. But when we blindly surrender to the menus we’re given, it’s easy

to lose track of the difference:

“Who’s free tonight to hang out?” becomes a menu of most recent people

who texted us (who we could ping).

“What’s happening in the world?” becomes a menu of news feed stories.

“Who’s single to go on a date?” becomes a menu of faces to swipe on Tinder

(instead of local events with friends, or urban adventures nearby).

“I have to respond to this email.” becomes a menu of keys to type a response

(instead of empowering ways to communicate with a person).

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 21/47

When we wake up in the morning and turn our phone over to see a list of

notifications — it frames the experience of “waking up in the morning” around a

menu of “all the things I’ve missed since yesterday.” (for more examples, see Joe

Edelman’s Empowering Design talk)

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 22/47

By shaping the menus we pick from, technology hijacks the way we perceive our

choices and replaces them with new ones. But the closer we pay attention to the

options we’re given, the more we’ll notice when they don’t actually align with our

true needs.

Hijack #2: Put a Slot Machine In a Billion Pockets

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 23/47

If you’re an app, how do you keep people hooked? Turn yourself into a slot

machine.

The average person checks their phone 150 times a day. Why do we do this? Are

we making 150 conscious choices?

One major reason why is the #1 psychological ingredient in slot

machines:intermittent variable rewards.

If you want to maximize addictiveness, all tech designers need to do is link a

user’s action (like pulling a lever) with a variable reward. You pull a lever and

immediately receive either an enticing reward (a match, a prize!) or nothing.

Addictiveness is maximized when the rate of reward is most variable.

Does this effect really work on people? Yes. Slot machines make more

money in the United States than baseball, movies, and theme

parkscombined. Relative to other kinds of gambling, people get

‘problematically involved’ with slot machines 3–4x faster according to NYU

professor Natasha Dow Schull, author of Addiction by Design.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 24/47

But here’s the unfortunate truth — several billion people have a slot

machine their pocket:

When we pull our phone out of our pocket, we’re playing a slot machineto

see what notifications we got.

When we pull to refresh our email, we’re playing a slot machine to see what

new email we got.

When we swipe down our finger to scroll the Instagram feed, we’replaying a

slot machine to see what photo comes next.

When we swipe faces left/right on dating apps like Tinder, we’re playing a

slot machine to see if we got a match.

When we tap the # of red notifications, we’re playing a slot machine to

what’s underneath.

Apps and websites sprinkle intermittent variable rewards all over their products

because it’s good for business.

But in other cases, slot machines emerge by accident. For example, there is no

malicious corporation behind all of email who consciously chose to make it a slot

machine. No one profits when millions check their email and nothing’s there.

Neither did Apple and Google’s designers want phones to work like slot

machines. It emerged by accident.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 25/47

But now companies like Apple and Google have a responsibility to reduce these

effects by converting intermittent variable rewards into less addictive, more

predictable ones with better design. For example, they could empower people to

set predictable times during the day or week for when they want to check “slot

machine” apps, and correspondingly adjust when new messages are delivered to

align with those times.

Hijack #3: Fear of Missing Something Important (FOMSI)

Another way apps and websites hijack people’s minds is by inducing a “1% chance

you could be missing something important.”

If I convince you that I’m a channel for important information, messages,

friendships, or potential sexual opportunities — it will be hard for you to turn me

off, unsubscribe, or remove your account — because (aha, I win) you might miss

something important:

This keeps us subscribed to newsletters even after they haven’t delivered

recent benefits (“what if I miss a future announcement?”)

This keeps us “friended” to people with whom we haven’t spoke in ages

(“what if I miss something important from them?”)

This keeps us swiping faces on dating apps, even when we haven’t even met

up with anyone in a while (“what if I miss that one hot match who likes me?”)

This keeps us using social media (“what if I miss that important news story or

fall behind what my friends are talking about?”)

But if we zoom into that fear, we’ll discover that it’s unbounded: we’ll always

miss something important at any point when we stop using something.

There are magic moments on Facebook we’ll miss by not using it for the 6th

hour (e.g. an old friend who’s visiting town right now).

There are magic moments we’ll miss on Tinder (e.g. our dream romantic

partner) by not swiping our 700th match.

There are emergency phone calls we’ll miss if we’re not connected 24/7.

But living moment to moment with the fear of missing something isn’t how we’re

built to live.

And it’s amazing how quickly, once we let go of that fear, we wake up from the

illusion. When we unplug for more than a day, unsubscribe from those

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 26/47

notifications, or go to Camp Grounded — the concerns we thought we’d have don’t

actually happen.

We don’t miss what we don’t see.

The thought, “what if I miss something important?” is generated in advance of

unplugging, unsubscribing, or turning off — not after. Imagine if tech companies

recognized that, and helped us proactively tune our relationships with friends and

businesses in terms of what we define as “time well spent” for our lives, instead of

in terms of what we might miss.

Hijack #4: Social Approval

We’re all vulnerable to social approval. The need to belong, to be approved or

appreciated by our peers is among the highest human motivations. But now our

social approval is in the hands of tech companies.

When I get tagged by my friend Marc, I imagine him making a conscious choice to

tag me. But I don’t see how a company like Facebook orchestrated his doing that

in the first place.

Facebook, Instagram or SnapChat can manipulate how often people get tagged in

photos by automatically suggesting all the faces people should tag (e.g. by

showing a box with a 1click confirmation, “Tag Tristan in this photo?”).

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 27/47

So when Marc tags me, he’s actually responding to Facebook’s suggestion, not

making an independent choice. But through design choices like this,Facebook

controls the multiplier for how often millions of people experience their social

approval on the line.

(Facebook uses automatic suggestions like this to get people to tag more people,

creating more social externalities and interruptions.)

The same happens when we change our main profile photo — Facebook knows

that’s a moment when we’re vulnerable to social approval: “what do my friends

think of my new pic?” Facebook can rank this higher in the news feed, so it sticks

around for longer and more friends will like or comment on it. Each time they like

or comment on it, we’ll get pulled right back.

Everyone innately responds to social approval, but some demographics

(teenagers) are more vulnerable to it than others. That’s why it’s so important to

recognize how powerful designers are when they exploit this vulnerability.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 28/47

Hijack #5: Social Reciprocity (Titfortat)

You do me a favor — I owe you one next time.

You say, “thank you”— I have to say “you’re welcome.”

You send me an email— it’s rude not to get back to you.

You follow me — it’s rude not to follow you back. (especially for teenagers)

We are vulnerable to needing to reciprocate others’ gestures. But as with Social

Approval, tech companies now manipulate how often we experience it.

In some cases, it’s by accident. Email, texting and messaging apps are social

reciprocity factories. But in other cases, companies exploit this vulnerability on

purpose.

LinkedIn is the most obvious offender. LinkedIn wants as many people creating

social obligations for each other as possible, because each time they reciprocate

(by accepting a connection, responding to a message, or endorsing someone back

for a skill) they have to come back to linkedin.com where they can get people to

spend more time.

Like Facebook, LinkedIn exploits an asymmetry in perception. When you receive

an invitation from someone to connect, you imagine that person making a

conscious choice to invite you, when in reality, they likely unconsciously

responded to LinkedIn’s list of suggested contacts. In other words, LinkedIn

turns your unconscious impulses (to “add” a person) into new social obligations

that millions of people feel obligated to repay. All while they profit from the time

people spend doing it.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 29/47

Imagine millions of people getting interrupted like this throughout their day,

running around like chickens with their heads cut off, reciprocating each other — 

all designed by companies who profit from it.

Welcome to social media.

After accepting an endorsement, LinkedIn takes advantage of your bias to

reciprocate by offering *four* additional people for you to endorse in return.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 30/47

Imagine if technology companies had a responsibility to minimize social

reciprocity. Or if there was an independent organization that represented the

public’s interests — an industry consortium or an FDA for tech — that monitored

when technology companies abused these biases?

Hijack #6: Bottomless bowls, Infinite Feeds, and Autoplay

Another way to hijack people is to keep them consuming things, even when they

aren’t hungry anymore.

How? Easy. Take an experience that was bounded and finite, and turn it into a

bottomless flow that keeps going.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 31/47

Cornell professor Brian Wansink demonstrated this in his study showing you can

trick people into keep eating soup by giving them a bottomless bowl that

automatically refills as they eat. With bottomless bowls, people eat 73% more

calories than those with normal bowls and underestimate how many calories they

ate by 140 calories.

Tech companies exploit the same principle. News feeds are purposely designed to

autorefill with reasons to keep you scrolling, and purposely eliminate any reason

for you to pause, reconsider or leave.

It’s also why video and social media sites like Netflix, YouTube or

Facebookautoplay the next video after a countdown instead of waiting for you to

make a conscious choice (in case you won’t). A huge portion of traffic on these

websites is driven by autoplaying the next thing.

Tech companies often claim that “we’re just making it easier for users to see the

video they want to watch” when they are actually serving their business interests.

And you can’t blame them, because increasing “time spent” is the currency they

compete for.

Instead, imagine if technology companies empowered you to consciously bound

your experience to align with what would be “time well spent” for you. Not just

bounding the quantity of time you spend, but the qualities of what would be

“time well spent.”

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 32/47

Hijack #7: Instant Interruption vs. “Respectful” Delivery

Companies know that messages that interrupt people immediately are more

persuasive at getting people to respond than messages delivered asynchronously

(like email or any deferred inbox).

Given the choice, Facebook Messenger (or WhatsApp, WeChat or SnapChat for

that matter) would prefer to design their messaging system to interrupt

recipients immediately (and show a chat box) instead of helping users respect

each other’s attention.

In other words, interruption is good for business.

It’s also in their interest to heighten the feeling of urgency and social reciprocity.

For example, Facebook automatically tells the sender when you “saw” their

message, instead of letting you avoid disclosing whether you read it(“now that

you know I’ve seen the message, I feel even more obligated to respond.”)

By contrast, Apple more respectfully lets users toggle “Read Receipts” on or off.

The problem is, maximizing interruptions in the name of business creates a

tragedy of the commons, ruining global attention spans and causing billions of

unnecessary interruptions each day. This is a huge problem we need to fix with

shared design standards (potentially, as part of Time Well Spent).

Hijack #8: Bundling Your Reasons with Their Reasons

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 33/47

Another way apps hijack you is by taking your reasons for visiting the app (to

perform a task) and make them inseparable from the app’s business

reasons(maximizing how much we consume once we’re there).

For example, in the physical world of grocery stories, the #1 and #2 most popular

reasons to visit are pharmacy refills and buying milk. But grocery stores want to

maximize how much people buy, so they put the pharmacy and the milk at the

back of the store.

In other words, they make the thing customers want (milk, pharmacy)

inseparable from what the business wants. If stores were truly organized to

support people, they would put the most popular items in the front.

Tech companies design their websites the same way. For example, when you you

want to look up a Facebook event happening tonight (your reason) the Facebook

app doesn’t allow you to access it without first landing on the news feed (their

reasons), and that’s on purpose. Facebook wants to convert every reason you

have for using Facebook, into their reason which is to maximize the time you

spend consuming things.

In an ideal world, apps would always give you a direct way to get what you want

separately from what they want.

Imagine a digital “bill of rights” outlining design standards that forced the

products used by billions of people to support empowering ways for them to

navigate toward their goals.

Hijack #9: Inconvenient Choices

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 34/47

We’re told that it’s enough for businesses to “make choices available.”

“If you don’t like it you can always use a different product.”

“If you don’t like it, you can always unsubscribe.”

“If you’re addicted to our app, you can always uninstall it from your phone.”

Businesses naturally want to make the choices they want you to make easier,

and the choices they don’t want you to make harder. Magicians do the same

thing. You make it easier for a spectator to pick the thing you want them to pick,

and harder to pick the thing you don’t.

For example, NYTimes.com lets you “make a free choice” to cancel your digital

subscription. But instead of just doing it when you hit “Cancel Subscription,” they

send you an email with information on how to cancel your account by calling a

phone number that’s only open at certain times.

Instead of viewing the world in terms of availability of choices, we should view

the world in terms of friction required to enact choices. Imagine a world where

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 35/47

choices were labeled with how difficult they were to fulfill (like coefficients of

friction) and there was an independent entity — an industry consortium or non

profit — that labeled these difficulties and set standards for how easy navigation

should be.

Hijack #10: Forecasting Errors, “Foot in the Door” strategies

Summary And How We Can Fix This

Are you upset that technology hijacks your agency? I am too. I’ve listed a few

techniques but there are literally thousands. Imagine whole bookshelves,

seminars, workshops and trainings that teach aspiring tech entrepreneurs

techniques like these. Imagine hundreds of engineers whose job every day is to

invent new ways to keep you hooked.

The ultimate freedom is a free mind, and we need technology that’s on our team

to help us live, feel, think and act freely.

We need our smartphones, notifications screens and web browsers to be

exoskeletons for our minds and interpersonal relationships that put our values,

not our impulses, first. People’s time is valuable. And we should protect it with

the same rigor as privacy and other digital rights.

Tristan Harris was a Product Philosopher at Google until 2016 where he studied

how technology affects a billion people’s attention, wellbeing and behavior. For

more resources on Time Well Spent, see http://timewellspent.io.

UPDATE: The first version of this post lacked acknowledgements to

those who inspired my thinking over many years including Joe

Edelman, Aza Raskin, Raph D’Amico, Jonathan Harris and Damon

Horowitz.

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 36/47

Written byTristan Harris

My thinking on menus and choicemaking are deeply rooted in Joe

Edelman’s work on Human Values and Choicemaking.

Featured In Entrepreneurship, Design, Social Media, Editor's Picks

Follow

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 37/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 38/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 39/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 40/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 41/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 42/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 43/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 44/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 45/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 46/47

5/27/2016 Tutorial: Step by Step Database Design in SQL

https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 47/47