In 2013, a Forbes opinion piece declared that Microsoft's Excel Might Be The Most Dangerous Software On The Planet. It’s been ten years since that declaration and Excel errors are still causing havoc for many organizations. Following are just a couple examples of how sloppy use of Microsoft Excel caused major headaches in the workplace.

Another example of Excel errors causing business chaos is documented in a November 2022 Bloomberg article.

These are just two examples highlighting what so many professionals already know. Microsoft Excel errors are very common, and the bad data can cause serious problems. In order to use Excel properly, it is important to understand why Excel has such a horrid reputation for major errors.

Microsoft Excel is one tool within the data management toolbox. Like any other tool, Excel can be (and often is) misused. Excel is a fantastic tool for examining data, for creating charts to visualize data distributions, and for evaluating datasets. These are the types of jobs that Excel spreadsheets perform very successfully.
 
In addition, Excel is also a very accessible tool to use when collecting information from remote users. Excel can be stored on common Sharepoint sites and used by multiple people in different locations to submit &/or view data. But the very accessibility of Excel, the ease with which one can share information using cloud platforms also leads to misuse.
 
For instance – in the example of employee recruitment given above, where some users were using a VLOOKUP list to populate Excel cells while other users were not, is a very common example of inconsistent data management. The information is being entered in different ways by different users across multiple spreadsheets. Because information is populated inconsistently, errors occur. Then when that data is merged, mistakes are often compounded and lost within the sheer volume of information in the master Excel file.
 
There is inconsistent data management in the second example as well, where some of the currency data included “foreign commas or amounts defined as text”. The first example explicitly states there were multiple users entering data in different ways, but there is also a very high probability that the irregular data in the second example is also a result of multiple users entering information in different ways.
 
So… what is the proper way to use in Excel in collecting data from multiple sources?

This is a very real problem because remote work is becoming increasingly common. The sheer ease with which organizations can throw a spreadsheet up on a Sharepoint site means the need for managing data integrity in shared applications will not go away.
 
For all the advantages Excel provides in ease of collecting and sharing information, this tool is not designed to store and manage large volumes of data. Nor is Excel designed to protect data integrity so that organizations are defended from major data errors. Within Microsoft’s Office Suite, the tool designed to store and manipulate data most efficiently and safely is Microsoft Access. Since Access is a database (and not a spreadsheet), it has built-in methods for managing data integrity that are simply not available in Excel.
 
Organizations set themselves up for major problems when they don’t use each tool appropriately. But it is possible to build top-notch data management solutions using Excel and Access together. Because Excel and Access are both within the Microsoft Office Suite these two tools can be paired to design very efficient in-house data management solutions.
 
For example, I’ve worked with clients to build combined Excel/Access applications for managing employee time reporting. Consider an organization where employees work from various locations and need to report their time to a central accounting department. In this example, let’s assume that each employee is servicing clients and, in reporting their time to accounting, they need to provide the following data points:

• Their own employee ID number
• Client ID (so the activity can be appropriately invoiced to clients)
• Date of activity
• Activity code (again to help with invoicing clients)
• Amount of time spent on the activity

 
In this situation one employee can enter several lines of billable time activities every single day. Each of those line-items will be used in the payroll process, in the client billing process and used for employee accountability. Because this information is important for so many processes, data integrity is essential. So, we must build a solution which is easily accessible in remote environments and also ensures data integrity.
 
A typical solution for the above problem would include the use of Excel for remote collection of employee time data. Then – at the end of a payroll period, after staff members enter their time, Excel data is uploaded into Access for processing in one central silo of information.
 
As a database tool, Microsoft Access has built-in capacities for ensuring clean data. Capabilities which are just not available in spreadsheets. For instance, data tables can be connected to each other using referential integrity restrictions. In concrete terms, this means that if a staff member enters a faulty ClientID into her spreadsheet – that error will be caught upon import into Access. This will happen because, with referential integrity restrictions in place, Access will not upload a time record if there is no corresponding ClientID in Access. The same can be said for data points such as Activity Codes, Employee ID numbers, etc…
 
The ability for Access to refuse import where matching key fields cannot be reconciled is essential for data integrity. Within Access it is possible to build reports which show records NOT imported because of data integrity issues. These reports can then be used to go back and fix wrong entries in the original Excel file or go back to the applicable employee to figure out why there is a discrepancy. Errors are caught in real-time and (most importantly) professionals using this information trust it because they see it being “cleaned” as data is imported into the Access side of the data management system.
 
In addition to cleaning data during the import process, within Access reports can be built for employees to review and confirm that their time is accurately imported into Access. Another advantage of using Access to upload, merge and store all employee time data is the benefit of compiling the data in different ways for different purposes. In this example, the information can be compiled for payroll purposes. Or the information can be used to invoice clients for billable time. Or… Activity Codes and time information can be used to assess and monitor either employee productivity or determine which services clients use most. And - because Access and Excel work so well together, once time data is centralized in a common silo and errors are cleaned out, it is possible to compile the data as needed and export it back to Excel for analysis or reporting needs.
 
By combining the use of multiple Microsoft products (in this case Sharepoint, Excel and Access), it is possible to use the best capabilities of each product to build a solution which works for everyone.
 
Microsoft dominates the office software market because they had the foresight to build the Office Suite so that information can easily move back and forth between different products. In this instance the ease of moving data from Excel to Access provides tools for building a data management solution which works both for gathering data from remote employees, but also cleaning the information as it is imported to a central database and ensuring long-term data integrity.

Michelle Meyer - 02/15/2024

​How many spreadsheets do you think are floating around on your workplace system? Does it make your head spin to even think about it? You’re not alone. Spreadsheet overload is a very common workplace issue.

​One reason spreadsheet proliferation is so common is because spreadsheets are being used to store data. Excel and other spreadsheet programs were designed to analyze information, not store it. Data storage is best managed with a database application.
 
However, in the nitty-gritty reality of real-world workplaces, information analysis and data storage are not so cut and dry. Often, there is specialized information which doesn’t quite “fit” into an organization’s enterprise level software. When this happens, Excel is an easy, convenient way to manage the specialized data.
 
Yet – there is a reason spreadsheets have such a bad reputation for errors. One very common cause for inaccuracies is the need to replicate spreadsheets over multiple reporting periods. Because spreadsheet data storage is so inefficient, users find themselves recreating workbook applications at the beginning of new reporting periods.
 
One example might be sales information where there is a different sales workbook for every year. When year-end arrives, the spreadsheet workbook is copied and saved as the new year’s application. Not only is this replication a huge contributor to spreadsheet overload; it also produces errors.
 
The process of copying/saving spreadsheets can break formulas and calculations. Also, if formulas and calculations are linked to other spreadsheets, replication can also destroy links and end up causing problems. If these breakdowns go unnoticed, or are not properly repaired, then mistakes occur. After numerous replications, it’s impossible to know where all the errors are and how extensive the problem is across the various spreadsheets.
 
Beyond the issue of error propagation, replicated Excel applications also create duplicate data sets. If – for instance – a network has multiple sales workbooks spanning several years, then common sales information (customer names, etc.…) is duplicated across all those spreadsheets. Not only is key information duplicated; spelling variations also proliferate. These types of duplicated, datasets are very difficult to effectively compile and study with any accuracy.
 
If these kinds of issues are showing up in your workplace, then it is time to consider evolving and migrating the offending spreadsheet applications to a database application. Microsoft created Excel to analyze data, but within its Office Suite, Microsoft also provides Access. Access is a database program that organizations use to house unique information which doesn’t quite “fit” into an enterprise-wide software program.
 
Like Excel, MS Access can be customized to an organization’s specific needs. MS Access is the best RAD (Rapid Application Development) tool on the market. So, it is possible to get an Access solution up and running in a timely manner, and at an affordable rate.
 
Since Access is part of the Office Suite, it is designed to work seamlessly with other Office Suite products. This means that when the time comes to analyze the data, it is quite easy to dump data to Excel for analytical purposes. But the data itself is stored and managed in a database application. This diminishes the Excel error rate dramatically, to say nothing of all the spreadsheets stored all over your network that everyone has lost track of.

This Forbes article might date back to 2013, but no one seems to be heeding the warnings. Spreadsheets are becoming notorious for errors. It’s not only a problem for our financial markets, but it also affects your organization, and your ability to make trustable decisions.
​
One of the biggest reasons spreadsheets have errors is a tendency for users to copy/paste data from one spreadsheet to another. This Forbes article discusses the copy/paste problem (and how it impacted our financial markets) in very stark terms.

It might seem a harmless task to copy/paste data from one spreadsheet to another, but it is not. Consistently, it will cause errors. Is it happening in your organization? The chances are high that it is.

It isn’t the tool that is the problem; it’s the way the tool is being used. Firstly, keep in mind one reality, when it comes to managing information. Data is dynamic and managing it requires dynamic solutions.

Look at your data through the lens of evolution. Since data is dynamic, proper management requires the ability to evolve and transform. Microsoft created its data management products to accommodate information evolution. The first stage of data evolution is the spreadsheet stage. One person may need to track data specific to his/her job and just starts a new spreadsheet. As time passes, and more people need access to the information, the original spreadsheet application grows and multiplies. Before long, users find themselves copy/pasting data from one spreadsheet to another. Your network ends up storing spreadsheets for every reporting period in folders all over the place and no one has a clue where all the data is at, or even if it is accurate anymore.

Spreadsheets are not designed to store data. Spreadsheets are designed to analyze data. Copying/pasting data from spreadsheet to spreadsheet is a classic sign of a data storage issue. When copying and pasting is occurring you need to move the data to a database solution. The second level of data evolution is the native, local database level. At this level, data from various sources are often combined and streamlined into one core native, local database.
​
Microsoft’s local database solution is Microsoft Access. Access is designed to store data efficiently, thus eliminating the need to copy/paste between spreadsheets. Access is capable of automating data processing – as was needed in the JPMorgan’s Chief value-at-risk model outlined in the Forbes article.
​
In addition, MS Access is the best RAD (Rapid Application Development) tool on the market. So, it is possible to get an Access solution up and running pretty quickly, and at an affordable rate. Since Access is part of the Office Suite, it is designed to work seamlessly with other Office Suite products. This means that when the time comes to analyze the data, it is quite easy to dump data to Excel for analytical purposes. But, the data itself is stored and managed in a database application. This diminishes the Excel error rate dramatically, to say nothing of all the spreadsheets stored all over your network that everyone has lost track of.

More Data Management Articles

• ​Microsoft Access and Data Integration
• Yes – Microsoft Access works in a Multi-User Environment
• YES – Microsoft Access Can be Used Securely
• Deploying Microsoft Access in a Remote Environment
• Why You Should Care about RAD and How It Impacts Your Bottom Line ....
• Do you have questions about your own data management project? Contact Michelle.

A few months ago, your co-worker retired, and you’re the one who ended up with all her spreadsheet applications. Yes, she took the time to teach you what she was doing with her spreadsheets, but it’s all pretty cumbersome and time consuming.
​
Just to maintain it all, you have to regularly export a lot of the data out of your proprietary records management system. Then, of course, there are all those columns of information that have to be hand entered, because your records management system doesn’t have the data.

In addition, the information in all those spreadsheets is related. So… beyond all the copying/pasting of data and importing information from your records management system, you also have to edit data across the multiple spreadsheets. If there is a change to the data, you have to make sure that change is reflected in every affected spreadsheet.

On top of it all, you didn’t create the spreadsheet system, and so you don’t always understand the logic behind it. This causes a lot of problems when you have to copy/paste/ and start new spreadsheets for new datasets, or new time periods. You’re concerned because there are so many different spreadsheets and so much copying and pasting of data, that there may be errors.

Worse, is the time you spend building reports from the data in various spreadsheets. Just trying to bring it all together, synthesize it in a way that reflects reality, and makes sense to those who want the report takes hours of your time.

Data is dynamic and managing it requires dynamic solutions

You’re not alone; this problem is quite common in offices. And there are productive ways to deal with it. Firstly, you’ll want to keep in mind one reality when it comes to managing information. Data is dynamic and managing it requires dynamic solutions

Look at your data through the lens of evolution. Since data is dynamic, proper management requires the ability to evolve and transform. Managing the information with spreadsheets may have made sense when your co-worker built them years ago. But, if you’re spending an inordinate amount of time maintaining them, or you’re concerned about data accuracy, then it is time to move to a new stage in data evolution.

Microsoft created its data management products to accommodate information evolution. The first stage of data evolution is the spreadsheet stage. One person may need to track data specific to his/her job and just starts a new spreadsheet. As time passes, and more people need access to the information, the original spreadsheet application grows and multiplies. Before long the situation has evolved into the system you inherited.

Actually this situation can be viewed as an opportunity. Since the spreadsheets have been passed from one owner to another, the potential to take your information management to the next stage in data evolution is high. The second level of data evolution is the local database level. At this level, data from various sources are often combined and streamlined into one core local database.

​Managing Local Data with a Database instead of a Spreadsheet
​
Local databases are unlike your system wide software solutions. Local databases are “local” to one group of users. These types of databases manage information that doesn’t quite “fit” into system wide proprietary software solutions. Local databases are a step-up from spreadsheet applications. When you move away from spreadsheets to databases, you gain the ability to manage your data in a more streamlined fashion. Databases allow you to organize data in such a way that you can eliminate duplicate data entry and the necessity to reproduce spreadsheets for various datasets. Database structure also allows for building standardized reports that can be used for multiple datasets.

Moving away from a spreadsheet solution, to a local database solution, will solve a lot of the problems you’re experiencing with your inherited spreadsheet applications. You’ll be able to manage related information in one core database, rather than distributed across many different spreadsheets. It’s still possible to import data from proprietary software. But, since databases are built off of a table structure, with relationship capabilities, you will be able to store imported data in the same database. Database relationships will eliminate the need copy/paste and start new spreadsheets for new datasets. With tables and table relationships the information can all be stored in the same database.

If a local database is properly built, the need for average users to understand database logic decreases. Properly built databases will dramatically reduce the need for average users to build and rebuild reports, or other output vehicles. Since standardized reports, exports, and datasheets can be built into the system and reused over and over again, only the person programming the database really needs to comprehend the underlying logic.

Database table structure allows for more control over data integrity, reducing duplicate records, and keeping your data clean. Spreadsheets are fraught with the capacity to produce errors. One of the reasons errors occur so much in spreadsheet applications, is because end-users copy/paste and reproduce the same spreadsheet “template” for multiple datasets. This dynamic can cause breakdown in data integrity when users forget to check calculations during the reproduction process. It can cause breakdown in data integrity when users forget to check for data duplication in the reproduction process. With a database solution, this problem is completely eliminated because proper database construction eliminates the need to reproduce databases for new datasets.

The Microsoft Office Suite is the most popular software on the market. This is because the Office Suite comes with different software packages for managing different office jobs. Microsoft Word manages word processing. Microsoft Outlook handles all the email and calendar tasks. Powerpoint is great for professional presentations. Excel is best used for data analysis. And Access is the database portion of MS Office Pro, it is best used for data storage and management. Within the Microsoft suite of products, Excel is the first stage in data management and Access is the next step up from Excel.
​
Microsoft gives most offices the tools they need for daily operations. But, Microsoft Access is the most underutilized portion of the Microsoft Office Suite. This can be a costly mistake for any office, because using Microsoft Access to collect, store and manage your local data can save you hours of time, a great deal of headaches with your spreadsheet applications, and it can save you money. 

Learn More About Microsoft Access​​

• Microsoft Access and Data Integration
• Yes – Microsoft Access works in a Multi-User Environment
• YES – Microsoft Access Can be Used Securely
• Deploying Microsoft Access in a Remote Environment
• Why You Should Care about RAD and How It Impacts Your Bottom Line ....
• Do you have questions about your own data management project? Contact Michelle.

When a Spreadsheet Makes Your Head Hurt it's Time to do Something Different:
​
One of the biggest reasons spreadsheets get out of control, is because they are not being used as they were designed to be used. To be specific, Microsoft created Excel to analyze data. Very often folks use Excel to store data. This is not what spreadsheet applications are designed to do.

For data storage, the best tool is a database. The best selling desktop database application on the market in Microsoft Access. Access complements Microsoft Excel very nicely. It is quite easy to move data back and forth between MS Access and MS Excel.

​So when should you move your data to a database environment?

Think "one-to-many". Specifically, learn to view your data through a relationship lens. All data is related. When data becomes too complex for an Excel Spreadsheet application, it is usually because the data relationships have become too complex. A typical example of a "one-to-many" relationship within a dataset can be seen in the diagram above. Characteristics to look for in this diagram are:

1. ONE Inventory item per line
2. MANY weeks of data for each inventory item. This can be seen in the columns titled: "Wk 1 Qty", "Wk 1 Cost", "Wk 2 Qty", "Wk 2 Cost"
3. To complicate things even more, observe the individual tabs for each week of inventory data

Now just follow this data structure design to its final conclusion. By the end of the year there will be one tab for each week of the year, and two columns for each week of the year. In addition, when one starts a new year, it will be necessary to start with a whole new spreadsheet application. In order to compare numbers from one year to the next, folks find themselves searching multiple spreadsheet applications and building an entirely new spreadsheet, where they paste data from previous years so they can effectively analyze the data. When you find yourself doing this kind of work to maintain spreadsheets, it's time to move to the next level of data management.

The second level of data evolution is the local database level. At this level, data from various spreadsheets are often combined and streamlined into one core local database. Just as Microsoft Excel is the "go-to" tool for simple data management and data analysis, Microsoft Access is the "go-to" tool for internal database systems. Access is most often used in that nebulous area where the data is too complex for a spreadsheet application, but does not quite fit into the larger system-wide proprietary databases.

It is this second tier of data evolution that is most often under-utilized. Most offices employ people who are comfortable in Excel. They can build and manage extensive Excel applications. In addition, most businesses spend a lot of money on system wide database applications at the 3rd level of data evolution. But, the second level of data evolution is often left without a "guiding hand". The information being tracked doesn't quite "fit" into the system wide database, nor can it be easily tracked with an Excel spreadsheet application. If the second tier of data is not being properly managed, users find themselves dealing with an uncontrollable situation. This is where Microsoft Access can solve a lot of problems.

Microsoft designed Access as local database tool. MS Access is capable of handling the complex "one-to-many" data relationships in a way that Excel spreadsheets aren't. In addition, in comparison to other database products, MS Access can be learned by lay people. If someone is familiar with Excel, they are most likely able to learn the basics of Access. If professional development is needed, it is usually still more cost-effective than custom software development with other database tools. 

​Michelle Meyer - 01/27/2018

More Data Management Articles

• Yes – Microsoft Access works in a Multi-User Environment
• YES – Microsoft Access Can be Used Securely
• Deploying Microsoft Access in a Remote Environment
• Why You Should Care about RAD and How It Impacts Your Bottom Line ....
• Do you have questions about your own data management project? Contact Michelle.