Document(s) 10 of 19

Introduction

Although the role of information programs and services on development is undeniable, there is little hard evidence linking information investments and levels of information activities to specific socioeconomic impact and development objectives. The IDRC program seeks to fill this gap by the development of models, based on valid qualitative and quantitative indicators, of information impact.

Following a series of workshops and discussions, a general framework for impact assessment, based on Griffiths and King (1993), was developed in a monograph (Menou 1993). The preliminary results reported here make more concrete some of the directions proposed in this book.

The general objective of the project is to develop and test a mathematical model that will indicate, for each of a set of input variables, their relative importance in accounting for the variation in a set of output or impact variables. As well, the model may incorporate latent variables, not directly observable, which mediate between the input and output variables. The main study is to investigate the impact of information on small businesses in Shanghai (China).

This sector of the economy was chosen because, as in most Western economies, it is currently booming. We have gained support and cooperation from the Institute of Scientific and Technical Information of Shanghai, where the actual data collection and analysis will be carried out in the fall of 1995. To gain experience with the methodology before using it in China, we conducted a pilot study in London, Ontario.

The London Project

The input variables we are interested in relate primarily to different sources of information used by those who are operating a small business. We need to identify, however, the other major factors that contribute to the variation in the impact (output) variables.

The output variables describe the success of the business, or its contribution to the local economy. The literatures of business, management, and information science contain few empirical studies specifically devoted to information factors in the success of small business. A number of studies have identified that factors such as innovation, know-how, creativity, and management competence play important functions in the success of small business (Chaganti, 1983). Other influences on business success include firm size, corporate status, and industrial sector (Tiggles and Green 1994).

In a survey of small manufacturing firms in Canada, Chaganti (1983) concluded that factors such as identifying a market niche and balancing quality with cost were also key factors for success. Clearly, many of these factors carry an information overtone. In fact, improving access to information, particularly through the Canadian network of Small Business Service Centres, was one of the key recommendations recently made to the Canadian government by a specially formed committee (Small Business Working Committee 1994).

To identify which variables were relevant to our pilot study and determine their potential interactions, we consulted with a number of experts: G. Stewart and J. Kinsella, Small Business Centre (London); P. Tripp, Head of Business Information Services, London Public Library (LPL); Prof. R. Knight, Western Business School; and Professor P. Kantor, Rutgers University. This consultation process lead to the design of a mathematical model to be tested and of a survey instrument used to collect information about these variables from a sample of small business owners in London.

Methodology

LISREL has been used widely in fields such as sociology and psychology. So far, there have been few studies in information science. For example, Auster and Lawton (1983) used a path-analysis approach to study the relationships between the interview techniques with users of online bibliographic retrieval systems, the amount of information gained by the users, and their ultimate satisfaction with the quality of the items retrieved.

In a different vein, path analysis was used to model the influence of various factors on the attitudes of users toward management information systems and their associated processes, products and services (Joshi 1992). Similarly, Baroudi et al. (1986) used questionnaire data to investigate the causal relationships between user involvement, usage of an information system, and user's satisfaction of the system using path analysis.

The LISREL methodology involves a number of steps:

• Identifying variables to be used,
• Collecting data on these variables,
• Developing the model,
• Testing the model against the data, and
• Revising the model if necessary and retesting it.

Data Gathering and Coding

The mailed package included a covering letter and self-addressed, stamped, return envelope. A second envelope was also included to allow participants who wished to receive a report of the main findings of the study to identify themselves. For ethical reasons, the participants were guaranteed anonymity and their identification on the questionnaire was optional.

The sampling frame consisted primarily of two databases of businesses in the London area, made available in electronic format to the researchers by the Economic Development Office of the City of London. A stratified sample of 919 small businesses was systematically extracted from the two databases (mounted on Microsoft Access, a database management software). This initial sample was augmented by the recruitment of 37 participants at a Small Business Fair (held in March 1994 at the Main Branch of the London Public Library - LPL), by scanning the Business Section of the London Free Press in the months of March and April 1994 (5 participants), by placing an ad in the HomePreneur, a local bulletin for small business owners (2 participants), and, finally, by directly distributing questionnaires to users of the Business Section at the Main Branch of LPL (19 participants).

From 21 April to 22 June 1995, 184 valid questionnaires had been returned. Another 28 had to be discarded because they were undeliverable (e.g., wrong address). Four returned questionnaires had to be excluded because they did not meet the definition of small business used in this study (50 employees or less). The adjusted return rate of 19.29% should be considered quite acceptable for a study of this type. Follow-up with the mailing list, had it been approved by the Ethics Review Board of the University of Western Ontario, might have improved the return rate.

Responses were anonymously coded and entered into an electronic data file using Microsoft Excel 5.0. Simple data screening (e.g., frequency distributions) was performed using PRELIS 2.12a (by Scientific Software International, Chicago, Illinois), a companion application software to LISREL. The frequencies for each answer are reported on the questionnaire in Appendix B. Values for all continuous variables were recorded as on the questionnaire. Answers to ordinal variables were coded: 1 for Never Use, 2 for Sometimes Use, and 3 for Frequently Use. Missing values were coded as 99 and answers that were not applicable were coded as 88.

Two of the most common problems with the data were answers in-between categories and multiple selection of categories for a given question (e.g., answer with both Sometimes Use and Frequently Use). These problems are partly because of the small number of possible answers given in the questionnaire (e.g., Not Important, Somewhat Important, Very Important). This can be remedied by providing more choices of answers in future questionnaires (e.g., Likert scale).

Some questions also proved to be too sensitive. For example, only 48 participants (26% of the total) chose to indicate their profit margin. The wording of some questions was also problematic. For example, the profit margin question (Question 7) was unclear as to whether we meant net or gross profit margin.

A more general issue concerns the respondents' position in their respective organizations and thus the reliability of the data collected. The questionnaires were sent out to the contact person identified by each firm in the sampling frame. In most cases, given the size of those businesses, these contact people will actually be the owner or the president (or other similar role) of the business. In a larger business, however, it is possible that the person filling out and returning the questionnaire may not have access to all company records. Although we can assume that a contact person has the authority to find out the correct information, the risk exists for incorrect information. This would be particularly problematic in the study of larger organizations in which case, the choice of the entry point would be a determinant for the quality of the data collected.

Data Analysis Using the LISREL Model

To illustrate the meaning of LISREL, let us consider the hypothetical model in Figure 1. This model states that business success ("success" oval in the graph) is caused by type of business ("type"), business environment ("busienvi"), and use of information ("info_use"). Each of these is a latent variable, which is measured by one or more observable variables. For example, the latent variable "success" can be measured by the profit margin ("PROFIT" rectangle in the graph), percentage increase in employees ("INCREASE"), and length of time in business ("LENGTH"). Lines pointing from independent latent variables to dependent latent variables are called paths.

Once data are collected on all the observable variables, LISREL will estimate the path coefficients that indicate the magnitude of the contribution of each independent latent variable to the dependent latent variable. In this example, path coefficients will allow us to assess the impact of information on business success when the contributions of other factors (e.g., business type and business environment) are considered together with use of information. LISREL will also tell us whether the path coefficient is statistically significant or not, that is, whether it is a true causation or is just a coincidence in the particular data we collected.

Findings: Path Analysis Models

It was soon realized that we do not have enough data points to test a model with so many variables. Thus, we deleted and combined some variables to achieve a more parsimonious that which has 10 observable variables and four latent variables as shown in Figure 3. For example, we deleted latent variable "type" and the corresponding observable variable "3 TYPES" because LISREL cannot deal with nominal data.

We deleted observable variable "PROFIT" because only 26% of people answered this question. Some variables were combined to form a composite variable by taking the arithmetic average of original scores. For example, latent variables "use of formal information" and "use of informal information" in Figure 2 were combined into a single latent variable "info_use" in Figure 3. This latent variable is measured by two observable variables "FORMAL" (use of formal information) and "INFORMAL" (use of informational information). Data on this combined variable "FORMAL" were obtained by taking the average of the five original variables (use of London Public Libraries, Small Business Centres, Internet, statistics, and newspapers, etc).

LISREL theory, however, and the test results of this revised model suggest that there are still too many variables in this version considering the number of data points we have (total 164 effectively used by LISREL). After several attempts at further combining variables and retesting the model, the model either did not converge, or converged into a nonadmissible point meaning that the model is not successful.

The estimated parameters (e.g., path coefficient, which will indicate the contribution of information to business success) produced by LISREL is not provided in this paper because these data are not meaningful when the model converged into a nonadmissible point. The path coefficients, therefore, on the Figures 2 and 3 are all zeros rather than the actual figures. The chi-square value and its significance level were not reported in this paper either for the same reason. We decided not to combine variables any further to fit the model because it will render a meaningless model from a research point of view, although the model might fit data statistically.

Experience Gained in the London Pilot Study

• Business success is an extremely complex phenomenon involving many variables. A simplified model using fewer variables will not capture the true causal relationships intended to be examined by Models with more variables, however, require significantly more data. Larger numbers of data points are need for ordinal data (data typically collected in survey questionnaires). Use of the LISREL model, therefore, is not recommended unless large amounts of data are available. Even larger amounts of data should be available if a questionnaire is used to collect data.
• Nominal data are not allowed in LISREL (i.e., at least ordinal scale data are needed). Thus "yes" or "no" questions, which are frequently used in questionnaires, should be avoided.
• Determining the appropriate output variables (indicators of business success in this study) is extremely important for the LISREL model to work. This is also a very difficult task. The lack of success of the London pilot study is partially because of the inappropriate output variables. For example, the variable "profit margin," which is probably one of the most important indicators of business success, was not included in the final version of model because an insufficient number of data points were available for this variable. (It is understandable that a lot of people chose not to answer this question because of its highly sensitive nature.)

Tentative Plan of the Shanghai Study

• Increase sample size. It will aim at 400 data points, which is more than twice that of the London pilot study.
• Include more effective business success variables, such as increase in fixed assets of the business.
• Limit the study to the manufacturing sector to reduce the factors involved and, therefore, simplify the model. It will also make comparisons of individual businesses more meaningful.
• Increase the return rate of sensitive data such as profit margin by collecting ordinal rather than ratio data. That is, rather than asking for the actual profit margin figure, we will ask interviewees to indicate whether the profit margin is excellent, good, satisfactory, etc. This of course will compromise the accuracy of the data in these variables. It is necessary, however, to ensure the availability of data.
• Pretest the questionnaire instrument to improve its quality, thus reducing the possibility of unclear questions. Increase the measurement scale for ordinal variables from three to seven (i.e., interviewees can choose from seven categories of answers rather than three).
• Increase the return rate. The questionnaire will be followed by a telephone call to encourage participation.

"Because this is an exploratory study, we cannot expect that the final result will be a clear cut determination of the exact, quantitatively determined, impact of information on the outcome variable" (Tague-Sutcliffe et al. 1994, p. 11). The difficulties we encountered in the London study confirmed this original estimation in our project proposal to IDRC. With our experience in London behind us, we hope to achieve a better result in the Shanghai study. Because of the difficult nature of this endeavour, however, we should not set our expectations too high.

References

• Auster, E.; Lawton, S.B. 1984. Search interview techniques and information gain as antecedents of user satisfaction with online bibliographic retrieval. Journal of the American Society for Information Science, 35(2), 90–103.
• Baroudi, J.J.; Olson, M.H.; Ives, B. 1986. An empirical study of the impact of user involvement on system usage and information satisfaction. Communications of the ACM, 29(3), 232– 238.
• Chaganti, R. 1983. A profile of profitable and not-so- profitable small businesses. Journal of Small Business Management, 21(3), 43–51.
• Griffiths, J.-M.; King, D.W. 1993. Special libraries: Increasing the information edge. Special Libraries Association, Washington, DC, WA, USA.
• Jöreskog, K.; Sörbom, D. 1988. PRELIS: A program for multivariate data screening and data summarization, 2nd ed. Chicago: Scientific Software International. Pp. 2-8.
• Joshi, K. (1992). A causal path model of the overall user attitudes toward the MIS function: The case of user information satisfaction. Information and Management, 22, 77–88.
• Menou, M.J., ed. 1993. Measuring the impact of information on Development. International Development Research Centre (IDRC), Ottawa, ON, Canada.
• Small Business Working Committee. 1994. Breaking through barriers: Forging our future. Industry Canada, Ottawa, ON, Canada.
• _____1995. Small business: A progress report. Industry Canada, Ottawa, ON, Canada.
• Stone, M.B. 1993. Assessment indicators and the impact of information on development. Canadian Journal of Information and Library Science, 18(4), 50–64.
• Tague-Sutcliffe, J.; Meadow, C.T.; Vaughan, L. 1994. Measuring the impact of information on development: A path analysis approach. A proposal to the International Development Research Centre (IDRC). IDRC, Ottawa, ON, Canada.
• Tiggles, L.M.; Green, G.P. 1994. Small business success among men-owned and women-owned firms in rural areas. Rural Sociology, 59(2), 289–310.

Appendix A: List of Variables

Appendix B: Questionnaire Used for Data Collection and Frequency Distribution

Measuring the Impact of Information on Small Businesses in London
A Questionnaire

[Only the first part of the questionnaire is included here. The second part was used to collect some qualitative information on the use of London Public Library services. These data have not been used in the study reported in this paper. This appendix also includes the frequency values (in %) obtained for each answer (total number of cases is 184). Note that the total for each question does not always add up to 100% because of missing values.]

Your responses to this questionnaire will be used to analyze information sources used by London small businesses. Your company will not be identified in this process.

I — Our first questions are about the size and scope of your business.

1. When did you start your business? [Median=1985]

2. What type of business do you have? Please, specify: _____________

[Manufacturing 10.6%

Retail 7.8%

Service 44.7%

Other 35.8%]

4. How many full-time employees did you have at start up? [Median=1]

5. Approximately how many hours of part-time employment did your business provide last year? [Median=275 hours]

6. Did you make a profit last year? [Yes 62.6%; No 32.4%]

If yes, do you mind telling us your profit margin? [Median=10%]

7. Do you expect to be in business a year from now? [Yes 93.9%; No 2.2%]

II — We are interested in the information you use in running your business.

1. How often do you use each of the following sources of information?

Source of information	Never use	Sometimes use	Frequently use
Friends, associates, relatives	2.8	46.4	48.6
Suppliers	7.8	40.8	49.7
Customers	5.6	41.3	52.0
Consultants,lawyers, accountants	16.2	63.1	19.6
Banks, other financial agencies	35.2	52.0	11.2
Newspapers,magazines, trade pubs	8.9	55.3	35.8
Stats Canada, other govt. pubs	60.9	29.6	7.8
Trade associations, business clubs	29.1	52.5	17.3
Internet	78.8	16.2	2.8
London Public Libraries	56.4	32.4	10.1
Small Business Centre	82.7	13.4	2.8
Other___________	12.3	2.2	6.7

2. Please tell us, for each of the following kinds of information, how important it is.

Kind of information	Not important	Somewhat important	Very important
Financing your business	21.2	33.5	43.0
Markets, customers	3.4	17.9	74.9
Suppliers	14.5	34.6	49.7
Government regulations	12.3	47.5	37.4
Management skills	11.7	34.1	51.4
Technology	7.8	31.8	58.7
Writing a business plan	30.2	45.3	21.2
Selling skills/motivation	14.0	35.2	50.3
Other_____________	7.8	0	3.9

III — What other factors have been important in determining the success of your business?

Factor	Not important	Somewhat important	Very important
Availability of financing	26.3	33.5	36.9
Location	38.5	35.2	23.5
Existing customer base	6.1	18.4	74.3
Technological developments	16.8	47.5	33.0
Exchange rate	60.9	24.6	12.8
Interest rate	40.2	35.2	22.3
Your expertise	0.6	8.9	90.5
Employees' expertise	11.2	15.6	69.8
Other_____________	4.5	0.6	6.7

Document(s) 10 of 19