The essence of the “expert survey” technique
What is an expert survey?
Expert survey is the collection of primary data based on the use of experience, knowledge and intuition of experts in the areas under study. Experts– specialists who know the specific aspects of the phenomenon being studied. In most cases expert interviews are conducted with representatives of the executive and legislative authorities of the regions, journalists from regional media, scientists, employees of universities and research organizations, employees of non-governmental, private expert or consulting structures, members of expert councils, etc. Expert survey has its own characteristics that distinguish it from mass surveys.
To conduct a survey, the interviewer must have sufficient competence in the subject being studied, as well as know the terminology used by professionals when discussing issues on the research topic. It is important to remain sensitive and polite. It is necessary to emphasize the importance of the opinion of each expert for the study, whereas in mass surveys respondents are informed that all data obtained will be subjected to general statistical processing.
As a rule, in expert surveys open language is used. Closed questions are asked extremely rarely, usually for clarification or to determine the extent of agreement with the opinions of other survey participants.
Purpose and capabilities of the “expert survey” technique
When should an expert survey be conducted?
The method is used in cases where there are no other methods of collecting data that satisfy the objectives of the study, and also when the object has specific characteristics that only professionals know about. Examples of research of this kind are, for example, projects devoted to estimating the volume of high-tech and specialized market sectors or latent (hidden) processes. Only a specialist can provide such information; mass survey methods turn out to be ineffective.
Methods expert assessments used to predict future events if statistical data is missing or insufficient. Expert assessments are also used to quantify events for which no other measurement exists, for example, when assessing the importance of goals and the preference of certain promotional methods. As expert is the person who is most competent on the problem being studied.
Usually, expert survey is aimed at clarifying hypotheses, developing a forecast and replenishing the interpretation of certain social phenomena and processes. In such surveys, open-ended formulations dominate, and closed-ended questions are intended only to assess the level of confidence, the degree of agreement or disagreement with the already expressed positions of other specialists.
Sample size for expert survey
How is the sample size for an expert survey determined?
Main selection criteria experts are their competence and authority, therefore the size and representativeness of the group of respondents in this case is assessed not so much by quantitative as by qualitative indicators.
3.4.4. Expert survey
A specific type of survey is an expert survey.
Expert survey – a type of survey in which the respondents are experts.
Expert – a specialist in any field of knowledge, involved in the study of certain issues that require specific competence. Experts to participate in the expert survey are selected, first of all, according to their level of competence; the size and representativeness of the group of experts is assessed not so much by statistical as by qualitative indicators. The reliability of assessments and decisions made on the basis of expert judgments is quite high and largely depends on the organization and focus of the procedure for collecting, analyzing and processing the opinions received.
An expert survey can be conducted either in the form of an interview or a questionnaire. These surveys are not anonymous, as they require the active cooperation of the respondent in clarifying the problems posed. As a rule, an expert survey is aimed at clarifying hypotheses: developing a forecast and replenishing the interpretation of certain social phenomena and processes. An expert survey is carried out to predict the development of a particular phenomenon, to assess the degree of reliability of a mass survey, to collect preliminary information about the research problem, in situations where a mass survey of ordinary respondents is not possible or effective.
Classification of expert survey.
1. By the nature of interaction between experts
- full-time – information is collected through personal communication with experts. The form of conduct is an informal interview. The advantages of this survey are that you can change the course of the interview depending on the expert’s answers and his competence;
- correspondence – the survey is conducted in written form.
2. By number of experts
- individual – only one expert participates in the survey. This type of survey allows you to obtain the most complete information on a specific expert;
- group – group discussion, brainstorming. The advantage of this type of survey is that direct clashes of different views are allowed.
Subject the survey is clearly defined in strict language, tasks research must also be clearly formulated, emphasizing the importance of the personal opinion of specialists (in mass surveys, on the contrary, they emphasize that the opinion of the respondent is taken into account within the framework of general statistics). In such surveys, open-ended formulations dominate, and closed-ended questions are intended only to assess the degree of agreement or disagreement with the already expressed positions of other specialists.
The expert survey procedure involves the following stages (Fig. 14):
Rice. 14 Stages of conducting an expert survey
3.5. Sociometric measurement method
The sociometry method is an effective tool for socio-psychological research into the structure of small groups and teams.
Term "sociometry" means the measurement of interpersonal relationships in a group. The totality of interpersonal relationships in a group constitutes, according to J. Moreno, the basis of sociometry, that primary socio-psychological structure, the characteristics of which largely determine the holistic characteristics of the group.
Sociometric technique is used to diagnose interpersonal and intergroup relations in order to change, improve and improve them. With the help of sociometry, one can study the typology of social behavior of people in group activities, and judge the socio-psychological compatibility of members of specific groups. The advantage of this method is that intragroup relations receive concrete expression in the form of tables, diagrams, graphs, and numerical values. However, all this information is not an exhaustive description of the group, since it represents only a description of existing interpersonal preferences, relationships of sympathy and antipathy. In addition, from the entire variety of informal relationships in the group, only those that are reflected in the wording of the proposed questions are identified. And finally, sociometry does not allow us to establish the motives for the selection or rejection of some group members by others. Therefore, it is usually used in combination with other methods for studying a small group or team.
Basic tasks solved by sociometry: measuring the degree of cohesion-disunity in a group; identification of “sociometric positions”, that is, the relative authority of group members on the basis of sympathy and antipathy, where the “leader” of the group and the “rejected” are at the poles; detection of intragroup subsystems - cohesive formations headed by informal leaders.
The sociometric procedure can be carried out in two versions. First option - nonparametric procedure . In this case, subjects are asked to answer questions without limiting the number of choices. The advantage of this option is that it allows us to identify the emotional expansiveness of each group member and make a snapshot of the diversity of interpersonal connections in the group structure. The disadvantage is the high probability of obtaining a random selection, as well as the inability to reveal the diversity of relationships in the group. Only the most subjectively significant connections can be identified.
Second option - parametric procedure with a limited number of elections.
One of the main concepts of sociometry is choice. Choice is a unit of measurement and analysis in sociometry. Expresses a person’s individual attitudes regarding interaction with members of his group in certain situations. It is an indicator of sympathy or antipathy. The main measuring technique of sociometry is question , responding to which each member of the group shows his attitude towards others. Depending on the question, sociometric choice can be positive (or direct), negative (or reverse) and zero (no choice).
The next concept is the sociometric criterion. Sociometric criterion – a specific situation of choice, which is formulated in the form of a written or oral question to all participants in the diagnostic procedure. The choice of sociometry criteria is dictated by the objectives of the study and follows from its program.
The following types of criteria are distinguished:
1. Depending on the nature of the research problem : communicative – aimed at identifying relationships in a group (for example, “Who would you choose for...”) and Gnostic – clarifying the degree to which a person is aware of his relationships with group members (for example, “Who, in your opinion, would choose you ...”).
2. By the nature of the relationships identified : double – presupposing partnership relations, equality of positions of the chooser and those chosen by him (for example, “Who would you agree to go with ....?”) and single – associated with the establishment of hierarchical relationships of leadership and subordination (“Who would you choose as the leader of the group?”).
3. By the nature of the answers : suggesting positive elections (like “Who would you choose for...?”) and offering negative choices (“Who would you reject if..?”).
4. By number of answers : nonparametric – without limiting the number of possible elections and parametric – with a clear limitation on the number of elections.
Primary requirements to the formulation of sociometric criteria are as follows:
The meaning of the questions should be extremely obvious to all members of the group, which requires the compiler to take into account the age, intellectual and other characteristics of the respondents;
All election situations must be described as specifically and accurately as possible (for example, the content of the criterion “Who would you like to work with?” needs to be clarified (where? when? in what capacity? under what conditions? etc.), otherwise there will be differences in the understanding of the phrase “work together” they will turn the question into one that means different relationships for different respondents;
It is necessary that the questions arouse some interest among the majority of respondents and have significance for them;
The wording should not contain unjustified restrictions on the selection of group members based on intellectual, sexual, physiological and other characteristics.
A questionnaire, a form of sociometric research is sociometric card – a means of obtaining information from respondents. It is here that individual elections are registered. If the criteria are not given to respondents orally, then a list of criteria is also posted here. It is recommended not to clutter the sociometric card with a large number of criteria, but to choose those that would be logically connected with each other and would arouse active interest in the majority of subjects. Sometimes the card also contains brief instructions for filling it out. When conducting a sociometric survey there is not and cannot be complete anonymity, otherwise the procedure itself will be ineffective. Therefore, the cards must be signed by the respondents. However, in some cases it is possible to use the method of hidden numbering of forms before the start of sociometry. If the researcher knows for sure which of the respondents filled out this form, the presence of a surname on it is not necessary.
The responses received from all group members for each criterion are combined into sociometric matrix – a table that summarizes the survey results. Analysis of the sociomatrix gives a clear picture of the relationships in the group. Based on the sociomatrix, it is possible to present the results more clearly in the form of a graphic representation of connections - a sociogram.
Sociogram is a graphical representation of the respondents’ attitudes towards each other when answering sociometric criteria. Allows you to more clearly express and deeply analyze group relationships, as well as to carry out a comparative analysis of the structure of relationships in a group in space on a certain plane using special signs.
The sociogram technique is a significant addition to the tabular approach in the analysis of sociometric material, as it allows for a deeper qualitative description and visual representation of group phenomena. The choice of sociogram type is determined by the objectives of the study.
Types of sociograms:
- arbitrary type – shows a combination of connections with the most convenient location of group members based on election results;
- concentric or “target” – shows the location of all members of the group on concentric circles inscribed within each other. The higher the positive status, the closer to the center of the circle a group member is located and vice versa;
- lokogram type - in which the subjects are designated in advance on the plane as they are actually located in the room where the main activities of the group take place.
Sociometric indices are used to identify quantitative characteristics of relationships in a group. There are indexes personal (P.S.I.), which give characteristics of individual socio-psychological properties of a person in the role of a group member and group (G.S.I.) - give numerical characteristics of the holistic sociometric configuration of elections in the group. The main P.S.I. are: an index of sociometric status (for the person chosen), an index of emotional expansiveness (for the person choosing) and an index of the adequacy of assessing one’s position in the group.
Interpretation of sociometric data is carried out by analyzing the data obtained during processing: sociomatrix, sociograms, sociometric indices.
The sociometric research procedure involves the following stages (Fig. 15):
Rice. 15 Stages of sociometric research
An expert survey is a type of sociological survey in which the respondents are a special type of people - experts. These are competent persons who have in-depth knowledge of the subject or object of study.
An expert (Latin expertus - experienced) is a specialist who makes a conclusion when considering a question.
A representative from any field other than our own can act as an expert for us. A distinctive feature of this method is that it assumes the competent participation of experts in the analysis and solution of research problems. For example, to assess the expected demand for certain types of products, experts can be sellers or store merchandisers, managers of small businesses, dealers or brokers. In an army unit, experts can be commanders, officers of educational structures, old-timers (they, as a rule, are knowledgeable on a wide range of issues of service and everyday life).
In this regard, the role function of the expert, who in the full sense of the word is an active participant in sociological research, is seriously changing. And an attempt to hide the purpose of the study from him, thus turning him into a passive source of information, is fraught with the loss of his trust in the organizers of the study2.
The expert method is distinguished from other forms of sociological survey by several very important features:
¦ the number of respondents: there are always fewer of them than during a survey or even an interview;
¦ the qualities of the respondents: their outlook, level of qualifications, knowledge of a special field are several orders of magnitude higher than that of ordinary respondents;
¦ type and volume of information: an expert survey is conducted to obtain knowledge that the sociologist-researcher does not and will never have; in contrast to ordinary knowledge, which is known to the sociologist from his own experience, the knowledge obtained from experts refers to special scientific knowledge;
¦ typicality of data: in a mass survey, a sociologist is interested in the typicality, repeatability, ordinariness of the information obtained about the value orientations and motives of people’s behavior, and in an expert survey, the researcher values the uniqueness of the expert’s technical or humanitarian knowledge, its depth, uniqueness;
¦ program function: the sociologist uses the primary information obtained in a questionnaire or interview to test scientific hypotheses, and in an expert survey - in order to understand an area that is completely new to him.
Such forms of collecting primary sociological information as questionnaires, interviews, postal surveys, and telephone interviews are intended primarily for mass surveys. Their peculiarity lies in the fact that they are aimed at identifying information that reflects the knowledge, opinions, value orientations and attitudes of respondents, their attitude to events and phenomena of reality. And the fact that this information is based on the individual interests of the respondents and can be very subjective does not at all contradict the scientific nature of its receipt. On the contrary, the purpose of a mass survey is to obtain reliable information about the subject and object of research using appropriate tools. For example, to identify groups of readers by the degree of their interest in various sections of the newspaper or to differentiate students by the level of their activity in class, etc.
Thus, during a mass survey, the source of sociological information assessing one or another aspect of the research object is representatives of the same object3.
The main purpose of the expert survey: identifying the most significant, important aspects of the problem under study, increasing the reliability, reliability, validity of information, conclusions and practical recommendations through the use of knowledge and experience of experts.
Scope of application of the expert survey: can be used to study all areas of activity; in diagnostics, in assessing the state of a social object, standardization, design, forecasting, and in decision making. Varieties of expert surveys are used quite effectively at different stages of sociological research: in determining goals and objectives, identifying problem situations, searching for hypotheses, interpreting concepts, justifying the reliability of tools and initial information, substantiating conclusions, and developing recommendations.
Basic regulatory requirements: when interviewing experts, it is necessary to provide a clear justification for the need to use the appropriate expert interview methodology. Careful selection of experts: mandatory assessment of their competence. Taking into account factors influencing expert judgments. Creating conditions for the most productive use of experts during the research. Preservation of information received from experts without distortion at all stages of the study.
There are some limitations in applying the findings to different expert survey techniques. Thus, when using some expert assessment techniques, it is useful to remember that the conclusions of their examinations tend to average opinions, and therefore may be unsuitable for assessing non-standard phenomena, for example, innovative works of art. We should also not forget that the data of an expert survey are subjective in nature, and therefore it is desirable to compare them with objective information about an object obtained by other methods (although often the use of an expert survey is caused precisely by the difficulty of obtaining information in another way).
The program for using an expert survey includes the main structural elements of a sociological research program. The leading tasks are: identifying the objectives of the survey, constructing selection criteria, rules for organizing the participation of experts and criteria for evaluating the information they provide. Unlike a mass survey, the expert survey program is not as detailed and is predominantly conceptual in nature. First of all, it clearly formulates the phenomenon to be assessed and provides possible options for its outcome in the form of hypotheses.
The main tool for expert surveys is a questionnaire or interview form developed using a special program. In accordance with this, the survey procedure can consist of either a questionnaire or interviewing experts.
Undoubtedly, to make informed decisions it is necessary to rely on the experience, knowledge and intuition of specialists. After the Second World War, within the framework of management theory, an independent discipline began to develop - expert assessments. Expert assessments are judgments of experts, involving a procedure for comparing objects and their properties according to selected criteria. The expert assessment method is a type of expert survey that involves the use of expert assessments. The main content of the method is the rational organization of the analysis of problems carried out by experts, followed by the assessment of the judgments identified by the researcher and the processing of the data obtained.
According to modern researchers, applied sociology is turning into a system of expert knowledge.
An expert survey is a collection of primary data based on the use of experience, knowledge and intuition of specialists in a particular field of activity. Most often, this method is used in applied sociology as a means of increasing the reliability of solving scientific and management problems.
A representative from any field other than our own can act as an expert. The distinctive feature of this method is that it involves competent participation of experts in analyzing and solving research problems. For example, to assess the expected demand for certain types of products, experts can be sellers or store merchandisers, managers of small businesses, dealers or brokers. In an army unit, experts can be commanders, officers of educational structures, old-timers (they, as a rule, are knowledgeable on a wide range of issues of service and everyday life).
The expert method is distinguished from other forms of sociological survey by several very important features.
- The number of respondents is always less than during a survey or even an interview.
- The qualities of the respondents - their horizons, level of qualifications, knowledge of a special field - are several orders of magnitude higher than those of ordinary respondents.
- Type and volume of information - an expert survey is conducted to obtain knowledge that the sociologist-researcher does not and will never have; Unlike everyday knowledge, which is known to a sociologist from his own experience, the knowledge received from experts refers to specifically scientific knowledge.
- Typicality of data - in a mass survey, a sociologist is interested in the typicality, repeatability, ordinariness of the information obtained about the value orientations and motives of people’s behavior, and in an expert survey, the researcher values the uniqueness of the expert’s technical or humanitarian knowledge, its depth, uniqueness.
- The program function - the sociologist uses the primary information obtained in a questionnaire or interview to test scientific hypotheses, and in an expert survey - to understand an area that is completely new to him.
Paradoxical but true. Russians are voting for a government that, as a Levada Center survey showed, they don’t like in the main thing—its disposal of the people’s money collected in the form of taxes. The survey was conducted June 21-26, 2012 among almost 1,600 people in 130 settlements in 45 regions of the country.
Such forms of collecting primary sociological information as questionnaires, interviews, postal surveys, and telephone interviews are intended primarily for mass surveys. Their peculiarity lies in the fact that they are aimed at identifying information that reflects the knowledge, opinions, value orientations and attitudes of respondents, their attitude to events and phenomena of reality. And the fact that this information is based on the individual interests of the respondents and can be very subjective does not at all contradict the scientific nature of its receipt. On the contrary, the purpose of a mass survey is to obtain reliable information about the subject and object of research using appropriate tools. For example, to identify groups of readers by the degree of their interest in various sections of the newspaper or to differentiate students by the level of their activity in class, etc. Thus, during a mass survey, the source of sociological information that evaluates certain aspects of the object of study are representatives of the same object.
The main purpose is to identify the most significant, important aspects of the problem being studied, increasing the reliability, reliability, validity of information, conclusions and practical recommendations through the use of knowledge and experience of experts.
Scope of application - can be used in the study of all areas of activity; in diagnostics, in assessing the state of a social object, standardization, design, forecasting, and in decision making. Varieties of expert surveys are used quite effectively at different stages of sociological research: in determining goals and objectives, in identifying problem situations, in searching for hypotheses, in interpreting concepts, in justifying the reliability of tools and initial information, in substantiating conclusions, in developing recommendations.
Basic regulatory requirements - a clear justification for the need to use the appropriate expert survey methodology; careful selection of experts (assessment of their competence is mandatory); taking into account factors influencing expert judgments; creating conditions for the most productive use of experts during the research; preservation of information received from experts without distortion at all stages of the study.
Limitations in applying findings to different expert survey techniques. Thus, when using some expert assessment techniques, it is useful to remember that the conclusions of their examinations tend to average opinions, and therefore may be unsuitable for assessing non-standard phenomena, for example, innovative works of art. We should also not forget that the data of an expert survey are subjective in nature, and therefore it is desirable to compare them with objective information about an object obtained by other methods (although often the use of an expert survey is caused precisely by the difficulty of obtaining information in another way).
An example of an expert survey within the framework of a research project of the Russian State Science Foundation for the regional competition “Russian North: history, modernity, prospects” G. V. Zhigunova conducted a sociological study aimed at identifying the attitude of various groups of the population of the Murmansk region towards children with disabilities .
The respondents were specialists from social institutions in the Murmansk region. The sampling is spontaneous, N= 210. In most cases, these are teaching staff of secondary educational institutions (68.5%), in addition, social work specialists, medical workers, speech therapists, speech pathologists, psychologists, etc. Of these, persons with higher education - 69.5%, with secondary specialization - 21%, with incomplete higher education - 5.7%.
Experts (47.6%) stated that morally and psychologically society is rather not ready to accept disabled children into its midst; 17.1% think society is not at all ready; 27.6% believe that society is somewhat ready, and only 1% believe that it is completely ready. The majority of experts believe that children with any disabilities should study in specialized institutions, and not in a regular school or at home. According to experts, today disabled children need day care centers, as well as a delivery service, access to medical care, the ability to purchase household appliances for children with disabilities, nurses with special education, and patronage.
The program for using an expert survey includes the main structural elements of a sociological research program. The leading tasks are to identify the objectives of the survey, construct selection criteria, rules for organizing the participation of experts and criteria for evaluating the information they provide. Unlike a mass survey, the expert survey program is not as detailed and is predominantly conceptual in nature. First of all, it clearly formulates the phenomenon to be assessed and provides possible options for its outcome in the form of hypotheses.
The main tool for expert surveys is a questionnaire or interview form developed using a special program. In accordance with this, the survey procedure can consist of either a questionnaire or interviewing experts.
Undoubtedly, to make informed decisions it is necessary to rely on the experience, knowledge and intuition of specialists. After the Second World War, within the framework of management theory, an independent discipline began to develop - expert assessments. Expert assessments - These are judgments of experts, involving a procedure for comparing objects and their properties according to selected criteria. Expert assessment method- a type of expert survey that involves the use of expert assessments. The main content of the method is the rational organization of the analysis of problems carried out by experts, followed by the assessment of the judgments identified by the researcher and the processing of the data obtained.
Background judgments record factors influencing the state of the object being studied. In the survey procedure, the expert identifies among them the most important, essential for the object, and gives them an assessment.
The use of the expert survey method is associated with compliance with certain rules. When organizing it, most attention is usually paid to three methodological problems - the selection of experts, the procedure for their work, and the processing of opinions expressed.
Let us list the errors and difficulties encountered when using an expert survey.
- 1. Unclearness in defining the objectives of the examination. Their inconsistency with the capabilities of the experts and the examination technique used. The expert survey is conducted without a specially developed program.
- 2. Ineffective use of experts. Expertise is designed to obtain information that could have been collected by other, less complex methods.
- 3. Insufficiently careful selection of experts. The assessment of the competence of candidate experts is one-sided (or absent altogether), which leads to the appearance of random people in the group of experts.
- 4. The experts involved in the survey do not show interest in the examination. Some experts are interested in distorting information.
- 5. The examination tools were not tested in advance. The questions are not formulated precisely enough, and their main meaning is not clear. There is a significant number of non-answers to the questions posed.
- 6. Experts experience difficulties in applying examination techniques.
- 7. Difficulties arose in interpreting the data due to the significant spread of answers, their heterogeneity, and the difficulty of comparing data in heterogeneous groups of experts.
- 8. During the examination, many quantitative estimates were obtained, but no attention was paid to their argumentation. Difficulties arose in explaining the data obtained. The average values of the answers turn out to be far from correct.
- 9. The desire to speed up the survey led to a superficial analysis of the problems by experts.
10. Excessive optimism in assessing the survey results. The examination data is not confirmed by other objective data
Expert survey- a type of sociological survey in which the respondents are a special type of people - experts. These are competent persons who have in-depth knowledge of the subject or object of study.
Expert(lat. expertus - experienced) - a specialist who makes a conclusion when considering a question.
A representative from any field other than our own can act as an expert for us. A distinctive feature of this method is that it assumes the competent participation of experts in the analysis and solution of research problems. For example, to assess the expected demand for certain types of products, experts can be sellers or store merchandisers, managers of small businesses, dealers or brokers. In an army unit, experts can be commanders, officers of educational structures, old-timers (they, as a rule, are knowledgeable on a wide range of issues of service and everyday life).
In this regard, the role function of the expert, who in the full sense of the word is an active participant in sociological research, is seriously changing. And an attempt to hide the purpose of the study from him, thus turning him into a passive source of information, is fraught with the loss of his trust in the organizers of the study 2 .
The expert method is distinguished from other forms of sociological survey by several very important features:
♦ number of respondents: there are always fewer of them than during a questionnaire or even an interview;
♦ the qualities of the respondents: their outlook, level of qualifications, knowledge of a special field are several orders of magnitude higher than that of ordinary respondents;
♦ type and volume of information: an expert survey is conducted to obtain knowledge that the sociologist-researcher does not and will never have; in contrast to ordinary knowledge, which is known to the sociologist from his own experience, the knowledge obtained from experts refers to special scientific knowledge;
♦ typicality of data: in a mass survey, a sociologist is interested in the typicality, repeatability, ordinariness of the information obtained about the value orientations and motives of people’s behavior, and in an expert survey, the researcher values the uniqueness of the expert’s technical or humanitarian knowledge, its depth, uniqueness;
♦ program function: the sociologist uses the primary information obtained in a questionnaire or interview to test scientific hypotheses, and in an expert survey - in order to understand an area that is completely new to him.
Such forms of collecting primary sociological information as questionnaires, interviews, postal surveys, and telephone interviews are intended primarily for mass surveys. Their peculiarity lies in the fact that they are aimed at identifying information that reflects the knowledge, opinions, value orientations and attitudes of respondents, their attitude to events and phenomena of reality. And the fact that this information is based on the individual interests of the respondents and can be very subjective does not at all contradict the scientific nature of its receipt. On the contrary, the purpose of a mass survey is to obtain reliable information about the subject and object of research using appropriate tools. For example, to identify groups of readers by the degree of their interest in various sections of the newspaper or to differentiate students by the level of their activity in class, etc. Thus, during a mass survey, the source of sociological information assessing one or another aspect of the object of study is representatives of the same object 3 .
The main purpose of the expert survey: identifying the most significant, important aspects of the problem under study, increasing the reliability, reliability, validity of information, conclusions and practical recommendations through the use of expert knowledge and experience.
Scope of expert survey: can be used to study all areas of activity; in diagnostics, in assessing the state of a social object, standardization, design, forecasting, and in decision making. Varieties of expert surveys are used quite effectively at different stages of sociological research: in determining goals and objectives, identifying problem situations, searching for hypotheses, interpreting concepts, justifying the reliability of tools and initial information, substantiating conclusions, and developing recommendations.
Basic regulatory requirements: When interviewing experts, it is necessary to provide a clear justification for the need to use the appropriate expert survey methodology. Careful selection of experts: mandatory assessment of their competence. Taking into account factors influencing expert judgments. Creating conditions for the most productive use of experts during the study. Preservation of information received from experts without distortion at all stages of the study.
There are some restrictions on use conclusions for different methods of expert surveys. Thus, when using some expert assessment techniques, it is useful to remember that the conclusions of their examinations tend to average opinions, and therefore may be unsuitable for assessing non-standard phenomena, for example, innovative works of art. We should also not forget that the data of an expert survey are subjective in nature, and therefore it is desirable to compare them with objective information about an object obtained by other methods (although often the use of an expert survey is caused precisely by the difficulty of obtaining information in another way).
Program application of an expert survey includes the main structural elements of a sociological research program. The leading tasks are: identifying the objectives of the survey, constructing selection criteria, rules for organizing the participation of experts and criteria for evaluating the information they provide. Unlike a mass survey, the expert survey program is not as detailed and is predominantly conceptual in nature. First of all, it clearly formulates the phenomenon to be assessed and provides possible options for its outcome in the form of hypotheses.
Basic tools expert surveys - a questionnaire or interview form developed according to a special program. In accordance with this, the survey procedure can consist of either a questionnaire or interviewing experts.
Undoubtedly, to make informed decisions it is necessary to rely on the experience, knowledge and intuition of specialists. After the Second World War, within the framework of management theory, an independent discipline began to develop - expert assessments. Expert assessments- these are judgments of experts, involving a procedure for comparing objects and their properties according to selected criteria. Expert assessment method- a type of expert survey that involves the use of expert assessments. The main content of the method is the rational organization of the analysis of problems carried out by experts, followed by the assessment of the judgments identified by the researcher and the processing of the data obtained.
Background judgments record the factors influencing the state of the object being studied. In the survey procedure, the expert selects among them the most important, essential for the object, and gives them an assessment.
The use of the expert survey method is associated with compliance with certain rules. When organizing it, much attention is usually paid to three methodological problems: the selection of experts, the procedure for their work, and the processing of expressed opinions 4 .
4.2. Types of expert survey
An expert survey is a combination (also called a complex) of various methods, techniques, techniques, and procedures. First of all, the procedure for the work of experts can be joint or separate. Among the collective procedures one can find the “brainstorming” method, regular discussion, and the Delphic technique. Let us highlight two main procedures: ordinary survey and multi-stage survey. The first involves conducting a one-time anonymous survey. It is both organizationally and economically the easiest. In principle, it is not much different from a regular mass survey. The second procedure is associated with the tendency to complicate the task of experts. Multi-stage is introduced so that at each subsequent stage experts solve increasingly complex problems. Typically, experts are asked to use various logical methods of analysis (“goal tree”, “mutual influence” tables, scenarios, etc.). The multi-stage survey itself can be organized in different ways: first, general questions can be asked, then more and more
specific (the “funnel” method) or, conversely, at the end the experts make some generalizations (the “pyramid” method).
Since survey participants are usually aware of the purpose and objectives of the survey, the meaning of using indirect questions, projective techniques, tests and other techniques that usually reveal the respondent’s positions without his knowledge is lost. Their use, as well as the use of “trap questions,” can even cause significant damage to the quality of the expert survey. After all, an expert is a participant in scientific research; any attempts to turn him from a subject of research into an object can shake the foundations of mutual trust, which is necessary between the organizers of the research and experts. In order to achieve an active and serious attitude from the expert, to make him feel like a full participant in scientific research, he must be introduced to some extent to the research program. Due to the specific nature of the expert audience, the main survey method is not an interview, but a questionnaire filled out by the expert himself. Moreover, in the questionnaire much more often they resort to open-ended questions, which make it possible to better establish the creative potential of the expert and enable the survey participant to express an original point of view. In addition, refusing cues weakens the influence of stereotypes.
Methodologists also distinguish methods of correspondence and face-to-face expert surveys. The methods of the first of them include: a written survey (gathering opinions), questionnaires (formalized survey), the method of independent characteristics and the Delphic technique, the second - an interview, meeting, research conversation, brainstorming. The number of experts should not exceed 10-15 people.
The simplest type of correspondence survey of experts is written survey(collection of opinions). It consists in the fact that specially prepared questionnaires are sent (distributed) to experts, in which they must express their opinion on the substance of the questions posed. When compiling an expert questionnaire, from 50 to 90% open-ended questions are used. Collecting opinions is similar to a free interview and differs from it only in the written form of the survey, which makes it possible to attract a large number of experts. True, the correspondence survey is associated with organizational difficulties due to the low rate of return of questionnaires.
Formal survey experts is a regular survey with questions formulated in both open and closed forms. In terms of goals, objectives and content, this method differs significantly from the written collection of opinions. If the latter is carried out to identify heuristic, fundamentally new ideas, views on the problem, unexpected sub-
moves to an old problem, then the survey is aimed at finding out assessments of certain aspects of the finished solution. The usual method of data analysis in this case is statistical.
Independent characteristics method allows you to give a generalized assessment of one phenomenon, information about which comes from several independent sources (from different people). At the first stage, different opinions are compared and contrasted, at the second stage they are processed using mathematical and statistical procedures, at the third stage reliable conclusions are formulated. This method is actively used in social psychology to study the business and personal qualities of an individual 5 . Here, several people who are well acquainted with the person being studied are asked to characterize him on a single scale, and then the independent assessments are summarized into an integral indicator. In a collective assessment, subjective deviations cancel out each other, which ultimately ensures an objective, scientifically based result.
One of the most common procedures for conducting an expert survey is "Delphic technique". The method involves interviewing experts in several rounds, processing the results of each round, informing them about these results and repeating the same procedure again. In the first round, answers are given without argumentation. After processing, extreme and average judgments are identified and reported to experts. In the second round, respondents return to their assessments. Since they have had enough time to think and learned about the existence of other positions on this matter, they are given the opportunity to reconsider their views or, on the contrary, argue. After the second round, new assessments are processed: extreme and average opinions are summarized, and the results are again reported to the experts. This is repeated 3-4 times. Practice shows that after the third or fourth round, expert opinions do not change. During such a procedure, a consensus assessment is developed, while the researcher should not ignore the opinions of those who, after repeated surveys, remained in their position.
The “Delphi” method is aimed at improving the process of accepting expert assessments, but with the goal that the overall assessment is derived not by mathematics and statistics, but by the people themselves, focusing on the opinions of others, if necessary, correcting it, strengthening their argumentation or abandoning it in favor of the best -
shay, more reasonable point of view. This technique makes it possible to reduce the influence of insufficiently competent experts on group assessment, as is observed with a simple questionnaire. This is achieved by obtaining valuable information from more competent experts.
Brainstorming method(“brainstorming”) is considered the most famous method of collective creative decision-making. It is a free, unstructured process of generating your own ideas around a given topic, spontaneously proposed by participants. The philosophy of this method is based on the assumption that with conventional methods of discussion, the emergence of innovative ideas is prevented by the control mechanisms of consciousness, which fetter the flow of these ideas under the pressure of familiar, stereotypical forms of decision-making. The inhibitory influence is the fear of failure, the fear of being funny, etc.
This method was developed and described by the American psychologist A.F. Osborne in 1938. The scientist drew attention to the fact that some people easily put forward new ideas, others are more inclined to critically analyze other people's ideas. In ordinary discussions, both categories of people end up together and, as a rule, interfere with each other. That is why it was decided to separate the stages of generating ideas and their analysis. For this, two groups are created: brainstorming participants - those who must propose new options for solving the desired problem, and commission members who will process the proposed materials. In the first group, which will generate ideas, a leader is first appointed who will staff this group, including 4-11 people. Members of this group cannot be bound by a “leader-subordinate” relationship due to the threat of destruction of the atmosphere of trust. The survey leader informs the group members of the essence of the problem being solved 2-3 days before the brainstorming session. The problem should be defined as clearly and comprehensively as possible. When conducting a brainstorming session, a relaxed atmosphere is created in the group. This is facilitated by the composition of the group, the behavior of the leader, the choice of premises, lighting, etc. None of the participants should be afraid that what they say is not valuable. Suggestions or ideas cannot be assessed negatively by word, gesture, or intonation (any criticism is prohibited); on the contrary, their support and development is desirable. Participants must express their thoughts openly and freely. This often reduces the stiffness of the group and is the root cause of the birth of original ideas. The proposals received during the discussion are recorded by the secretary. Mozgovoy 494
the assault lasts no more than 2-3 hours. Too quick completion is undesirable, since it has been established that new and original ideas often arise when it seems that all possible thoughts and associations have been exhausted. I
Two forms of brainstorming are considered common: simple meeting And round-robin meeting.
In a simple meeting, the manager interviews each participant one by one and asks them to make a proposal for solving the problem they are facing. Each decision is listed and numbered, then the list is posted in plain sight. Criticism or evaluation of ideas is not permitted. Particular importance is placed on creating a free and creative environment that allows all experts to freely express their ideas and suggestions. The number of proposals submitted is of great importance. Everyone must participate in their nomination. Impromptu is especially valued, i.e. proposals that arose immediately and under the influence of the information that the expert heard from others. They are valued above homemade preparations because they accumulate collective thought, multiplied by knowledge of the situation and the creative imagination of the author of the idea. If the attack on the unknown is too sluggish, the meeting is postponed to another date, allowing the experts to “ripe.”
In a round-robin meeting, experts are divided into small groups of 3 or 4 people, where they generate new ideas and write them down on a piece of paper or cards (2-3 ideas each). Small group members then exchange their cards, resulting in new ideas being added to old ones. After three exchanges, each subgroup compiles a consolidated list of ideas put forward. Then the entire team gathers, for whose consideration group reports are submitted. This form is advisable to use when activity decreases or when participants are distracted while waiting for their turn.
The list of ideas put forward as a result of brainstorming is usually quite long (more than 15-20). It can be difficult for the presenter to decide on their priority, and for the participants to wait for their turn to discuss. It is recommended to use the following method to help. A list of ideas with serial numbers is posted in a visible place. Each expert is entitled to five votes, which he can use at his own discretion: one vote for each of five ideas, all five for one, two votes for one idea and one for each of the other three, etc. This approach allows each expert to express his preference, and the team as a whole to decide on priorities. Another way: on
At the group meeting, each idea is read out under its own number, and experts vote by show of hands. The number of extended fingers on a raised hand indicates the number of votes cast 6 .
Method "reverse brainstorming" In many ways it resembles a regular brainstorming session, but at the same time it is allowed to express critical remarks. More precisely, criticism is specifically encouraged, since the philosophy of this method is built on the fact that all experts identify as many shortcomings as possible in the proposed ideas. The method can give good results if it acts as a preliminary procedure to other examination methods.
Method forecast scenarios- the most popular method of expert assessments in recent decades. The term “scenario” was first used in 1960 by futurist G. Kahn when developing pictures of the future necessary for solving strategic issues in the military field. A scenario is a probabilistic description of a picture of the future, compiled on the basis of competent technical judgment. One forecast includes several scenarios, in most cases three: optimistic, pessimistic and intermediate (most likely, expected). Drawing up a scenario occurs in several stages: 1) structuring and formulating the question: collecting and analyzing initial information, agreeing on the task with all project participants, identifying the structural characteristics of the problem; 2) identification of external influence factors; 3) finding indicators, preferably alternative ones, of the future state; 4) formation and selection of consistent sets of assumptions using computer programs; 5) development of practical recommendations for the future scenario and determination of the possible consequences of its implementation.
Method collective notepad(“bank” of ideas) is a method based on a combination of independent proposal of ideas by each expert with their subsequent collective assessment.
Method KJ- This is the name of a method of anthropological research in which researchers first collect a collection of facts about the life of a tribe, and then ask the natives to explain their meaning. Japanese business has adapted the method KJ as follows: company employees are asked to jot down on pieces of paper their wishes for improvements in the production process and suggestions for what the company should do. The received wishes and suggestions are analyzed, and based on the sum of opinions, a picture is obtained that depicts the prospect of the company-496
institute and its divisions in the future. The method is more of an integrating rather than analytical nature.
Method layman lies in the fact that the solution to the problem is offered to people who have never dealt with it, but are specialists in related fields.
Example. At the Center for the Sociology of Regional and National Relations of the ISPI RAS (headed by Corresponding Member of the RAS V.N. Ivanov), the study of problems of interaction between the federal center and the regions is based on systematic surveys of expert groups 7 . The expert groups or so-called groups of specialized consciousness included: specialists from regional (republican, regional) and city administrations, heads of enterprises and institutions of various forms of ownership, workers in the media, higher education, as well as representatives of creative unions . The method of collecting information is a handout questionnaire. The geography of research was very wide. Using a Comparable methodology, studies were carried out in Moscow, Stavropol, Ufa, Petrozavodsk, Yakutsk, Ulan-Ude, Tyumen, Novosibirsk, Kazan, Astrakhan, Tambov, Saransk, Ryazan, Rostov-on-Don, Volgodonsk, Barnaul, Vladikavkaz, Nalchik, Nazran. Data from expert surveys conducted in 2000-2002 indicate that if the prevailing situation by the end of the 1990s. the situation in the Russian Federation could be characterized as a crisis, and in the early 2000s. it begins to improve (a positive trend in expert assessments over time was discovered). Annual expert surveys conducted by the Department of Social Ecology of Regions of the ISPI RAS (headed by I. A. Sosunova) make it possible to identify environmental problems that are significant for the population and give rise to negative socio-ecological processes. In particular, it has been established that the problems perceived most painfully by the population are those pro- appearing in the healthcare sector and generating socio-demographic, socio-economic and other consequences. As a result of a survey of members of the expert councils on social and human sciences of the Higher Attestation Commission, employees of the Department of Theoretical Problems of Sociology of the ISPI RAS (headed by L.N. Moskvichev) received the following data: almost half of the experts note a decrease in the requirements for candidate works and about 40 % of experts note a decrease in requirements for doctoral work. Approximately the same situation arises in relation to dissertations in the sociological sciences.
4.3. Selection of experts
\In both mass and expert (in contrast to mass, it can be called elitist) polling, almost the main point is the selection of respondents. In the first case, we are talking about the correct compilation of a sample population, which will subsequently ensure representativeness of the data. The level of education and competence, if this is not a survey of the target group, do not play any role - anyway, the questions are standardized for everyone. In the second case, the problem is almost the opposite. Typical experts are not always needed. Often a sociologist needs reference, the best representatives of their profession. And no one will check the data for representativeness. The larger the population of respondents, the higher the reliability of data in a mass survey as some average statistical indicators. As for the expert survey, due to the high competence of the people participating in it, the opinion of even one expert, and even more so a group of experts, may turn out to be quite justified and reliable. Let us also note another fact: some technical and methodological techniques widely used in mass surveys lose their significance when surveying such a specific audience as experts. As a rule, mass surveys are anonymous. In expert surveys, this loses its meaning, because experts must be fully aware of the tasks that are being solved during the study with their help. For example, the expert questionnaire does not use indirect and control questions, tests or any other techniques aimed at identifying the “hidden” positions of the respondent.
The composition of the expert group determines the effectiveness of using this method. The fact is that such characteristics as knowledge and experience on this issue can only be used at the very initial stage of selection. Very often, a researcher includes everyone who works in a given field in the initial list of potential experts, and then selects a narrow circle of people from it.
How to competently form the necessary group of experts? This is what the authors of the well-known book in our country, “Fundamentals of Applied Sociology,” propose, published under the editorship of a prominent methodologist, director of the Center for Social Forecasting and Marketing F.E. Sheregi 10. At the very first stage of selection, it is advisable to use two criteria as criteria: occupation
and work experience in the profile we are interested in. If necessary, the level, nature of education, experience in social and political activities, age, etc. are also taken into account. The first list of experts can be very broad, but later it is advisable to “narrow it down”, since not every person is capable act as an expert.
The central criterion for selecting experts is their competence. To determine it, two methods are applicable, with varying degrees of accuracy; self-assessment of experts And collective assessment of the authority of experts.
The simplest and most convenient form of self-assessment of experts is a cumulative index, calculated based on the experts’ assessment of their knowledge, experience and abilities on a ranking scale with the positions “high”, “medium”, “low”. In this case, the first position is assigned the numerical value “1”, the second - “0.5”, and the third - “0”. In this case, the cumulative index - the coefficient of the expert’s level of competence - is calculated using the formula:
Where k l- the numerical value of the expert’s self-assessment of his level of theoretical knowledge; to 2 - numerical value of self-assessment of npaKii tic experience; to 3-numerical value of self-assessment of the ability to make a forecast.
The competence level coefficient can vary from (full competence) to 0 (complete incompetence).
Typically, it is customary to include in a group of experts those with in-deck competence of at least average (0.5) and higher (up to 1). Obtaining primary numerical values of self-esteem (k v kj, k 3) for The expert competence index is calculated using a tabular question (Table 4.1).
Based on the numerical values of the assessments indicated by crosses in the cells of the table, we will calculate the coefficient of the level of competence of an expert on the problems of the rule of law:
The resulting number indicates that the expert’s level of competence on the issue under study is above average.
When selecting experts using the self-assessment method, the problem of overestimation arises. However, as experts note, “the experience of numerous examinations both here and abroad shows that groups with high self-esteem, as a rule, are less mistaken in their judgments during examinations than others” 11 .
The collective assessment method is used to form a group of experts when they know each other as specialists. This situation is most often typical for scientists, creative figures, famous politicians, and economists. This method can be considered using the example of the formation of a group of experts carried out by S. Beshelev and F. Gurvich.
Let's assume that we have a list of ten experts and ask each of them to identify the five most competent colleagues. Based on the answers, we will build a table. 4.2, designating in it with numbers from 1 to 10 in the first column those who are chosen, and in the first line those who choose. In the cells of the table, the number “1” indicates a choice, a “dash” - no choice, “0” - indicates that no one named themselves.
Table 4.2 Table of mutual assessments of experts
| Who was named | Who named | How many times did I call | |||||||||
| - | - | - | - | ||||||||
| - | |||||||||||
| - | - | - | - | - | - | - | |||||
| - | - | - | - | ||||||||
| - | - | ||||||||||
| - | - | - | - | ||||||||
| - | |||||||||||
| - | - | - | - | - | - | - | |||||
| - | - | - | - | ||||||||
| - | - | - | - | - | - | ||||||
| Total |
The last column of the table shows the sum of votes that the corresponding expert received. These numbers are taken as the “weight” of opinions, they are replaced by “units” in the table (in columns), resulting in a ranked series of assessments of the level of competence of all ten experts (Table 4.3).
Table 4.3 Table of updated mutual assessments of experts
| Who was named | Who named | Total score | Rank assessment of I competence | |||||||||
| - | - | - | - | - | ||||||||
| - | - | |||||||||||
| - | - | - | - | - | - | - | - | |||||
| - | - | - | - | - | ||||||||
| - | - | - | ||||||||||
| - | - | - | - | - | ||||||||
| - | - | 1". J | ||||||||||
| - | - | - | - | - | - | - | - | |||||
| - | - | - | - | - | ||||||||
| - | - | - | - | - | - | - |
Thus, if we want to form a group of the five most competent experts, then, judging by the estimates in the penultimate column, we will enroll experts numbered 7, 5, 2, 9, 4 into this group.
Surveys of competent persons are called expert surveys, and the survey results are called expert assessments. In the most general form, we can distinguish two main functions of the expert assessment method in sociological research: assessment of the state (including causes) and forecast of development trends of various phenomena and processes of social reality.
4.4. Factors validity expert assessments
Validity is the trust that expert judgments deserve. It depends on the competence of the experts and the difficulty of the problem being solved. There are many reasons why an expert may not be able to realize his intellectual and creative potential. Let us note some of them and give recommendations for eliminating them.
1. Identification of a judgment with its source- a natural component of human perception by a person. Significantly removed in the absence of direct contact of experts. If
If the group works in face-to-face interaction mode, it should be composed in such a way that it does not include people who know each other well, bosses and subordinates. Including “big names” in a group is advisable only when the entire group consists of high-ranking specialists.
2. Centripetal pressure arises due to the tendency of most people to shift their judgments towards the neutral middle. It has been established: the higher the self-assessment of the expert’s competence, the higher the stability of the expressed judgments. The transition to correspondence interaction between experts does not in itself exclude the emergence of centripetal pressure, however, some procedures for collecting information (procedures with qualitative feedback, with individual feedback) make it possible to cope with this complexity.
3. Desire for dominance may arise among members of the expert group who are not recognized leaders; this is a potential source of conflict. Inclusion of experts with a strong desire for leadership in direct impact surveys should be avoided.
4. The instability of the emerging opinion typical for most experts at the stage of getting to know the object and initially thinking about their assessments. An immature judgment expressed at this moment may be perceived by others as a welcome hint. At the first stage, it is necessary to maintain information isolation of experts.
The reliability of expert assessments is increased due to the correct formalization of the initial information, the correct selection of experts, and compliance with the principles and procedures of expert questioning. It also depends on the individual qualities of the expert (professional and practical knowledge, length of service and experience in a specific field of activity, etc.).
Loading...
