One of the tasks of a database developer is to gather information about a project or a proposed project in order to craft a database design.
The first step is naturally to learn as much as possible about the project as it relates to storing and retrieving data. Projects are often much larger than just using a database, but for now we will concentrate only on issues related to the database, what types of data it needs to stored and how it might be used.
First, a few terms to get started.
We normally refer to the client as the person, group or organization who wants the project done, or who will be the owner of the project. This could be a friend, your boss, another department where you work, or another company or organization. The client may be one person or a group. It may be a group with a leader, or several people who are loosely organized. (This should not to be confused with another use of the term "client", which can refer to a desktop or other device used to run a user application.)
A development team is a group of developers and/or other people who work on tasks related to a project. The "team" might actually be a single person. A project may be tackled by a single person, a development team, or multiple teams.
Sometimes, different parts of a project are handled by different people or different teams. Larger projects often have a "project manager", whose job is coordinate how the project is defined, executed, and kept on track.
For smaller software projects, a single developer might handle the various parts of the effort. This might include the user interface (UI or front end), the middleware or server application – if there is one, and the database development (or backend).
For large projects, it is common to have separate developers, or separate teams to handle these aspects of the project. There may even be a separate team to gather and document requirements, a team to prepare graphics and other visual components, and a team to test and document the application.
For now, we will simply refer to whoever is performing the project work as the "development team", even though it could be a single person.
A stakeholder is anyone (person, group, organization) that may be involved in a project, including people that have input into the project design, or people that may only be impacted by the project. Some stakeholders are easy to identity. For example, if a company is creating a Shipping application, stakeholders include the shipping manager, the dock workers or material handlers, and truck drivers.
Some stakeholders are less obvious. In this example, what about the security department and the maintenance department? What about accounting and human resources? What about companies that ship to this company, and those that receive shipments?
Why are stakeholders important? Three main reasons:
- projects are more successful when they involve all the people that may be impacted by the project – positively and negatively
- if a project does not take into consideration how major stakeholders are affected, it might fail because it is not useful or actually harmful to what it was trying to achieve (for some stakeholders)
- not all stakeholders are supportive; some might actually oppose or set out to derail a project
The point of examining these various roles is that managing projects is not always a simple and straight-forward process. The larger the project, the more people and stakeholders are involved, and the more complicated the process. The graphic below illustrates the role of the project manager, who answers to the client, coordinates all the other actors, and attempts to manage the project plan, budget, timelines, etc.
Learning About the Project
How do we go about learning more about a project and what is needed? There are many possible approaches.
One starting point is to simply listen to what the client is proposing and try to understand the project objectives. The client may have a very detailed concept of what they want, or it may be very general. For example, the client may want to sell cars and has lots of ideas about keeping inventory and taking orders. Or, perhaps the client simply wants to advertise yard sales, without any specific ideas of how to go about it.
In some cases, a client may have existing paperwork or files that are used in their current process and that describe what the database must track. Spreadsheets, digital or paper forms, reports, and other files may be handy in understanding what types of data the project needs to collect and/or to report on. Although not as common, the client may also have work flow diagrams, information flow diagrams, or may have already prepared wireframe or other visual examples of what the application needs to do and what data needs to be collected, etc.
A "wireframe" is a graphical example of a data entry form or report. The wireframe example below suggests several types of data that need to be collected for an application.
Sometimes, a client wants to create an application similar to an existing application. For example, the client might want to create a web site to sell gardening tools, and there are two or three other sites that have features that they are interested in replicating. This does not necessarily reveal much about what the database needs to look like, but it provides good clues to the types of information that may be important to the project.
For larger projects, another way to collect requirements is to survey stakeholders and those people who will be most involved. For example, if there are twenty people who take orders for a company, do they all take orders the same way? What things would be most important to them in a new application? What types of things might make their jobs easier, or harder?
Questionnaires are the best method for gathering opinions and other information from many people.
Discussing a project in a group can sometimes bring out issues and spark ideas better than talking with people one-on-one, and better than using questionnaires and other survey methods. Such groups might be informal meetings or might be professionally guided sessions, such as Six-Sigma - a formal quality process. In some cases, groups might be used to identity possible solutions, to review potential solutions, or to critique prototypes and new designs.
For any size project, it sometimes helps the design or development team to see how things actually work. If the project is for a new web order app, for example, how do users actually select products, decide what to order, and complete the order process? How does customer service assist users with questions? What are the most common questions? How are those issues resolved?
Observation is one of the most time-consuming methods to gather requirements. However, even when other methods are used to gather and define requiements, observation may also be used to confirm the processes and information flows that are identified by other methods.
For example, a client may have a well-defined and documented order process. The managers and workers may explain in detail how it works. They may even have work flow diagrams and other documentation explaining the existing process, such as the workflow diagram example above. But by observing the actual processes and information flows, the design/development team may find out that the process does not work exactly the way management thinks it does, not how it is documented, nor how workers describe it. This means a new application might fail at critical points because it does not include the undocumented steps that the process includes. In the example above, what if managers often skip the phone interview and approve loans without talking to the loan applicants? How good is the database information then? How does the company identify and prevent fraud by unworthy borrowers?
It is common for issues to arise in a project that no one is certain how to handle. Perhaps the client would like to use QR codes to track products and make information available to customers. What types of printers and software are needed? What are the costs? Is this technology easy to implement for the company's products? These are basic questions that may need to be answered before the project can move forward, so research may need to be done to get detailed information before a good picture of the project can be created.
Research may need to be performed in many stages of the SDLC. As requirements are collected, there are always issues of
- what is possible?
- what is practical?
- what is cost effective?
- what will be acceptable? (to management, employees and customers)
- what is ethical
- what is legal
Even in the design and build (coding) stages, there may be questions of
- what are the common industry practices?
- what are best practices?
- what is technically feasible?
All of these issues may require research before decisions can be made about the best way to proceed.
Evaluating Project Information
As project information is collected, it is important that the development team evaluate what they have and try to identity what they do not have.
The evaluation must consider the following:
- Clients are not always aware of how their own organization works. The larger the organization, the larger the possible disconnect between what management thinks happens, and what actually happens on a day-to-day basis.
- Some information is more important and relevant, and some information is useless or not pertinent. While it is possible to gather lots of information, it is also important to weed through and identify what is important to the current project, and what can be mostly ignored. Take the shipping project: the vice-president of the company might spend a great deal of time talking about the size and cost of the new warehouse, the type of air-conditioning it has, the upgrades made to the shipping dock, the number of people hired in the last few years, etc. – none of which may be relevant to the proposed shipping application.
- Sometimes organizations do not follow their own rules about how things work. For example, perhaps the rules say that every outgoing package should be weighed, but in reality, clerks estimate the weight and write down their guesses – because it is faster.
- It is also true that most clients are not software engineers or database experts. They know a lot about what they do, but they may not understand how to translate that into requirements for software and databases.
- Unfortunately, some people will also shade the truth, or outright lie. This is something the development team must be aware of. For example, perhaps a clerk will greatly inflate how much time a task takes, or severely underestimate the importance of a task. Why? It may not have anything to do with the project at hand. Perhaps the employee feels overworked and thinks that exaggerating the time it takes will support requests for more help (or money). Or maybe the new application sounds like a huge hassle and the employee just doesn't want to be bothered with it.
The point to all of these issues is that it is not likely the client will hand the development team a perfect list of requirements. It is up to the development team to evaluate and question whether the information they have is sufficient, reliable and credible.
Starting a New Project: The Project Scenario
It is common for the development team to meet with a client to learn more about what is needed. The team may, or may not, have some advance documentation and other information describing the project, the project goals and objectives, and details about the desired features for the software.
Regardless of which type of situation is encountered, there is often too much information and too little information (at the same time). One necessary step is to analyze the existing information and pick out what is important (and what is not important).
This exercise is meant to simulate the initial development team meetings with the client. The purpose is to practice gathering and evaluating information in order to derive the project requirements.
Assume you are part of a development team. Read the following sample scenario about a new project the team has been assigned. Instructions about how to review and evaluate the information are provided after the scenario.
Analyzing the Scenario
In this scenario, the focus is on a business that provides housing and dog-sitting services. If someone on the team is very familiar with this type of business, or has worked on projects with similar businesses, they may have special insights on what to look for.
Regardless, the initial challenge is to identity what types of information appear to be most important and relevant to the software project.
As a first step, it is helpful to do two things:
- Mark out any information that seem unimportant
- From what is left, highlight points in the scenario that seem especially important as it relates to collecting and using information
Re-Analyzing the Scenario
Let’s consider the scenario again, marking out and highlighting some of the scenario information.
Reasoning Behind the Evaluation
The goal in evaluating the scenario is to eliminate information that is not useful and then highlight things that seem to be the most relevant to the task at hand (building a web site and database).
We already know that the project is to build a web site and a database to support it; we don’t really need that information. How long the company has been in business, the fact that it moved, etc., are not particularly relevant.
How the company currently keeps records could be of special interest. Forms and reports, and things such as Excel files, illustrates what the organization does now – and most likely will need to do in a new application. There is also mention of an "intake form", which would provide details about how data is collected now.
The types of data that the company currently tracks are of special interest. Those are things the new database will likely need to do.
The specifics of what the company does, in terms of housing dogs, and how that works, is also of special interest. An estimated number of appointments will be very useful to building a database big enough for this application.
Information about supplemental activities (feeding, walking, other care) describe things related to appointments.
How payments are collected is especially important, if the web app needs to also do that.
The fact that the client has identified a "special problem" suggests the development team should also consider is a special requirement.
There are at least two issues where the client has indicated things may change or be done differently, which is very important to the development team, because these are issues where new data need to be collected, and the process of how it might work is not yet known.
Synthesizing and Summarizing
How could we summarize the evaluation of the project so far? Perhaps we need to provide a short summary of the meeting to our project manager. An outline will suffice for now:
- Company houses and cares for dogs only
- An intake form, Excel, and other paper forms are used to track data
- Primary data collected is for customers, appointments and dogs
- Customers may have multiple pets
- Appointments are 1-3 days
- Estimated 24-30 appointments per day, most for 2 days
- Special instructions are tracked for foods, feeding, medicines, walking, and other care
- Names of permitted people to pick up each dog are captured at intake
- Customers pay by credit card at pickup time
- Company may open another location
- Payments may have an option to be billed in the future
Identifying Entities from the Evaluation
From the evaluation, and from the outline summary, is it possible to identity sets of data (entities) that need to be tracked?
Actually, three things were already mentioned: customers, appointments and dogs.
It seems like there "might" be a need to track special instructions, per dog or per appointment.
It is not yet clear, but there is also a question about whether payments (or bills) need to be tracked.
Here is a list of possible entities:
- special instructions?
Explicit and Implicit Requirements
If we feel we have a good understanding of the project so far, it is time to dig a little deeper. The client actually provided a lot of useful information. In some cases, the details tell us exactly what is needed. These are explicit requirements. In other words, the client has attempted to tell us exactly what is needed. For example, we need to keep track of customers, appointments and dogs. This is both helpful and important. It is important because it is a key thing that the client wants. If we build an application that does not keep track of appointments, we have failed to understand the client's directions.
But not all requirements are explicit. Some are unclear, missing or implicit. Implicit requirements are ones that seem obvious based on other information, or are derived from other requirements, but that are not clearly stated – yet.
For example, from the scenario, it seems implicit that the application needs to also track things like food choices, medicines, other care activities, pickup people, as especially payments. These are all related to appointments – but they sound like they could also be separate items.
There are also some questions about whether payments will be collected differently.
Perhaps most important, is the possibility that Pawsit might open up another location. That suggests that each appointment might need to be linked to a location; something that would be very important for the database designers to know.
Lastly, there are some possible requirements that simply have not been mentioned. For example, is there any possibility that Pawsit will start accepting other pets, such as cats and pet pigs? Also, should all employees have the same access to data in a new application? Who can see payments and payment information? These are potential requirements that have not been mentioned.
The focus on implicit and unstated requirements is important in a software development project. Why? A software application is sometimes like building a house. It is relatively easy to change the color and paint the house a different color after it is built. But if after the house is built, the owner says: "oh, I forgot we wanted the basement to be fifteen feet high instead of ten" – making that change is almost impossible to make. Being aware of all requirements, even potential future requirements, is important to make sure the application and database are prepared correctly.
For a software application, an issue such as having multiple locations for the pet store, may be critical to the overall design. Being aware of such requirements at the beginning of the project means they can be planned for up front - even if it is not implemented immediately. Changing key requirements after the application is built is much harder to implement. It benefits the development team to be alert for implicit and unstated requirements that may significantly impact their work.