Investigation of Subjective Acoustical Attributes by ...

ACTA ACUSTICA UNITED WITH Vol. 95 (2009) 1060 – 1070

ACUSTICA DOI 10.3813/AAA.918238

Investigation of Subjective Acoustical Attributes by Performers Through Ranking Data Analyses Gianfranco Genta1) , Maria Giovannini2) , Arianna Astolfi3) , Giulio Barbato1)

1)

2)

3)

Politecnico di Torino, Department of Production Systems and Business Economics, Corso Duca degli Abruzzi, 24, 10129, Torino, Italy. [email protected] Istituto Nazionale di Ricera Mettrologica (I.N.Ri.M), Thermodynamics Department, Strada Delle Cacce, 91, 10135, Torino, Italy. [email protected] Politecnico di Torino, Department of Energetics, Corso Duca degli Abruzzi, 24, 10129, Torino, Italy

Invited paper based on a presentation at the ICA 2007 in Madrid Summary This paper focuses on the acoustic perception of professional musicians performing on a stage or in an orchestra pit. A questionnaire was used to obtain a ‘priority classification’ of acoustical attributes to describe their requirements. The Borda count was chosen as the analysis method. As it is not suggested for subjective data ascribed to ordinal scales, it has been validated through a comparison with the more rigorous and suitable Condorcet method and a modified version of this method. The uncertainty assessment was taken into account and the Normalized Error was used to compare the results related to five orchestra-concert hall configurations and one orchestra-opera house configuration. The factors that influence the ranking have been investigated and the different acoustical conditions in the auditorium do not seem to have affected the answers. The results suggest the existence of possible orchestra and stage effects or other unknown systematic effects which were not considered in the final classification. The attributes pertaining to the accuracy of ensemble playing and the playing technique were judged to be more important than those related to the stage or pit acoustics, even though the quality of the playing aspects depends on the acoustics. PACS no. 43.55.Hy, 43.75.Cd, 43.75.St

1. Introduction Only a few studies have been conducted to investigate the acoustic conditions preferred by musicians, whereas much more effort has been dedicated to the audience. Since 1978, research has focused on which subjective attributes represent the acoustic conditions of musicians in a performance space and measurements of objective parameters have also been carried out with the aim of obtaining significant relationships [1, 2, 3, 4, 5]. Marshall et al. [1] emphasized the importance of the early reflections for “ensemble” in small orchestra. Naylor and Craik [2], through experiments conducted in simulated sound fields, found that ease of hearing themselves and others was necessary to perform in a synchronous way and to be able to tune correctly. They correlated the Modulation Transfer Function for simple signal-to-noise type degradation to the balance between the hearing of one’s own instrument and the hearing of the other instrument(s). Meyer [3], summarizing the factors which constitute the acoustic issues relevant to orchestral players, stated that most of these factors can

Received 28 March 2008, accepted 27 July 2009.

1060

be influenced by the surrounding hall. For instance, the duration of the decay time of musical instruments is influenced by reverberation time (where string and woodwind instruments need a longer reverberation time as their decay time is short while that of the piano is long and therefore takes advantage of its own reverberation) or the impression of dynamics and the timbre are supported by the strength of overtones, which can be distorted by excessive absorption at high frequencies. Gade [4] carried out interviews with professional musicians in order to establish which acoustical attributes concern music players. He established six relevant subjective attributes: “Reverberance”, “Support”, “Timbre”, “Dynamics”, “Hearing each other” and “Time delay”. “Support” was associated with the sensation of musicians hearing their own instrument without effort, while “Hearing each other” and “Time delay” were associated with the ease of ensemble. After surveys conducted with five orchestras [5], he proposed the support parameters, STearly , STlate and STtotal , which measure different portions of the reflected energy relative to the direct sound, to evaluate the subjective responses of “Ensemble”, “Reverberance” and “Support”, respectively. Some studies have distinguished conditions for small chamber music orchestras from those of large symphony orchestras; in the latter the cross-stage attenuation reduces

© S. Hirzel Verlag · EAA

Genta et al.: Investigating subjective acoustical attributes

ACTA ACUSTICA UNITED WITH ACUSTICA

Vol. 95 (2009)

Table I. Type of orchestras and survey features. Name of the orchestra Rai National Symphony Orchestra

Orchestra type

Auditorium

Number of respondents

Resident

G. Agnelli Auditorium

44

Rai Auditorium

23

Turin Philharmonic Orchestra

Touring

G. Verdi Conservatory Orchestra of Turin

Touring

Sanremo Symphony Orchestra

Resident

Sanremo Casino Theatre

20

Teatro Regio Orchestra of Turin

Resident

Teatro Regio of Turin

41

the relative level of direct sound with relevant implications on sound delays and the level of reflections. Ueno et al. [6], focusing on chamber music performances, found the ease of “Making harmony” essential as well as “Hearing each other”. They found that, in an ensemble, sounds made by different instruments need to be properly separated in order to sound in harmony, and this is obtained with an optimal energy level of reverberation that is neither too weak nor too strong. The survey carried out by Sanders [7] had the aim of investigating chamber musicians’ perception. In her study, “Support” was found to be the most highly correlated attribute with “Overall Acoustic Perception”, and this was followed by “Balance”, “Ensemble” and “Reverberance”. As can be deduced from the above review on the state of the art, confusion can arise on what the most important subjective attributes are for musicians playing on a performance space (stage or pit). With the purpose of knowing more about their subjective perception, professional orchestra players were asked to rank most of the acoustic subjective attributes given in literature in order of importance (“priority classification”). Some of the possible causes of influence on the responses derived from the musicians’ backgrounds and the acoustics of the performance space where they usually play, have been considered in the paper.

2. The orchestras Five professional Italian classical orchestras participated in the survey after musical rehearsals. Four of these orchestras usually play in concert halls while the fifth is an opera orchestra which usually performs in an opera pit. One orchestra, the Rai National Symphony Orchestra (RNO), took part in the appraisal after it had played in the G. Agnelli Auditorium (AA) and the Rai Auditorium (RA), two of the most important concert halls in Turin (Italy). The RNO has always played in the G. Agnelli Auditorium since 1994 and on 19 January 2006, just four months before the second survey, it went back to play in its original restored hall, the Rai Auditorium. The Turin Philharmonic Orchestra (TPO) and the G. Verdi Conservatory Orchestra of Turin (VCO) are tour-

G. Verdi Conservatory Concert Hall of Turin

9 15

ing orchestras which rehearse in the G. Verdi Conservatory Concert Hall of Turin (VC), while the Sanremo Symphony Orchestra (SSO) is a resident orchestra which plays in the Sanremo Casino Theatre (ST). The Sanremo Casino Theatre is a small concert hall with a stage and a fly tower, without a pit, which is also used for opera and drama. The Teatro Regio Orchestra (TRO) is the resident orchestra of the Teatro Regio of Turin (TR), which is an opera house occasionally used for concerts, musicals and ballet. Table I lists the orchestras, the auditoria where they rehearsed before the survey and the number of respondents, while some dimensional and acoustic data of the performance spaces, including the mid-frequency reverberation time, T20 , and the support parameters, STearly and STlate [5] on the furnished stage or pit, are reported in Table II.

3. The questionnaire The questionnaire consists of a list of acoustical attributes which have to be ranked in order of importance. There are nine attributes for each of the case studies with the exception of the RNO which performs in the Agnelli Auditorium and the TRO which performs in the Teatro Regio, for which a further attribute was included. On the basis of the aforementioned subjective surveys for the musicians, the following attributes were chosen: • “Clarity” (CLA), • “Reverberance” (REV), • “Dynamics” (DYN), • “Sound Envelopment” (SENV), • “Sound Strength” (SS), • “Tempo” (TMP), • “Own Instrument Perception” (OIP), • “Tonal Balance” (TB), • “Timbre” (TBR), • “Ensemble” (ENS). “Ensemble” was only added for the RNO in the Agnelli Auditorium and for the TRO in the Teatro Regio. For the other orchestras, the authors erroneously considered that the importance of “Ensemble” was so taken for granted that it was not even included in the list. The musicians

1061

ACTA ACUSTICA UNITED WITH Vol. 95 (2009)

ACUSTICA


Table II. Features and performance space data. CH = Concert Hall, OH = Opera House. The T20 , STearly and STlate values are the average results in the octave bands 0.5–1 kHz, 0.25–2 kHz and 0.25–2 kHz, respectively. All the measurements were carried out in unoccupied, furnished stages or a pit according to a protocol described in [19]. * from suspended panels. ** within the proscenium arc. *** up to the proscenium arc.

Performance space G. Agnelli Auditorium Rai Auditorium G. Verdi Conservatory Concert Hall of Turin Sanremo Casino Theatre Teatro Regio of Turin

Type

Area [m2 ]

24400 14500

CH CH

244 ∼190

15.0 9.9*

Yes Yes

6700 ∼3400** ∼11000**

CH CH OH

∼100 ∼140 ∼100

10.7 ∼16.0 13.4***

Yes No Yes

were asked to assign a number from 1 (the most important) to n (9 or 10, the least important) to the attributes. The number of the chosen attributes was probably too high and there was a risk of their meanings being overlapped. However, the aim was not to establish a functional essential set of attributes to describe the performance space from the musicians’ point of view, but to find an order of importance among those proposed in literature. The meaning of ‘importance’ of an acoustical attribute cannot be completely unambiguous. An attribute could be considered important because during his/her career an orchestra member has experienced substantial variation in the underlying attribute, or because it is easily related to concepts used in musical language. Furthermore, an attribute becomes important (or unimportant) because the orchestra member that usually plays in a hall is conscious of a lack or an excess of this attribute, or because the musician has always experienced an optimal condition of the attribute. Issues connected to the musician’s background and language were considered during the preparation of the questionnaire discussed below, while the other questions, which depend on the hall where the musicians play, relate to the questionnaire administration and are discussed in section 5. The influence of the musician’s background is difficult to deal with and constitutes a source of uncertainty. As stated by Ueno and Tachibana [8], a musician’s judgment is influenced by his/her skill, motivation and career, therefore their evaluation of acoustic conditions can be very different and can vary according to the type of performance (e.g. solo, in a quartet or in an orchestra). In the present case, the questionnaires were anonymous and there were no questions on musicians’ background, except for one concerning on the instrument they played. This information was requested as it had been intended to compare the answers from different instrumental groups, but the groups were too small to carry out elaborations separately, and the whole orchestra judgment was therefore considered. As far as musical language is concerned, the meaning of the words used to express perception is not necessarily shared by all musicians or between musician and acousticians. In order to overcome this problem, important information was collected during face-to-face interviews and

1062

Mean Height [m]

Stage or Pit features Susp. refl. T20 [s]

Volume [m3 ]

STearly [dB]

STlate [dB]

2.0 1.8

−16.1 −15.2

−17.5 −15.5

2.0 1.6 1.3

−11.2 −16.5 –

−11.4 −15.1 –

an agreement was reached about the words that were used. For example, the attribute “Clarity” was employed in the questionnaire, but for a musician this word cannot be related exclusively to a clear perception of sound, e.g. the music of a pipe organ in a large church is not clear, but it is clear to a musician since it was composed for this type of space. For this reason, the meaning of the attributes was explained before the questionnaire was handed out to avoid misunderstandings. “Clarity” was described as the ability to clearly perceive the music phrasing; in this way it refers to the clarity of the musical message which depends on the music genre. According to some musicians, it can affect the tuning and be influenced by reverberation. “Reverberance” refers to the degree of perceived reverberation and, according to the musicians, a hall which is acoustically too dry does not allow some instruments, e.g. the woodwinds, to act on the dynamics, making the sound weak and “sour”. “Dynamics” refers to the dynamic range that can be obtained in the auditorium, that is the perception of the ff and pp, the crescendo and diminuendo. “Ensemble”, which is related to the ease of hearing each other and playing together, is also influenced by reverberation, according to some musicians. “Sound Envelopment” is related to the spatial aspect of the perceived sound and is interpreted by musicians either in a positive way associated with reverberation, or in a negative way as a lack of sound directivity which leads to a lack of clarity. “Sound Strength” refers to loudness, “Tempo” is associated with the ability to keep the tempo and rhythm without delays, while “Own Instrument Perception” refers to the perception of the musician’s own instrument among others, which is the same as Gade’s “Support” [5]. “Timbre”, for most of the performers, is usually applied to the quality of a musician’s sound, but it was here indicated as the property of the room to influence the timbre. “Tonal Balance” concerns the balance between high and low frequencies. The criteria for self and other perception (e.g. clarity of “my” instrument and clarity of other instruments at “my” position) were not specified but, as indicated in the questionnaire, the musicians interpretation concerned the general perception of the sound on the stage or in the pit.



Vol. 95 (2009)

Table III. The questionnaire used in the survey. Italian version

English version

Indagine sull’acustica degli spazi destinati ai musicisti

Survey on the acoustics of spaces for musicians

Quale strumento suona?

Which instrument do you play?

A quale delle seguenti caratteristiche attribuisce maggiore importanza? Le elenchi in ordine di priorità mettendo dei numeri da 1 (il più importante) a 10 (il meno importante).

Which of following attributes do you give more importance to? List them in order of importance, numbering them from 1 (the most important) to 10 (the least important).

Nitidezza (o Chiarezza) Si riferisce alla capacità di distinguere in modo nitido l’articolazione della frase musicale

Clarity Refers to the ability to perceive the music phrasing, in a clear way

Riverberazione Si riferisce al grado di riverberazione dell’ambiente

Reverberance Refers to the degree of the perceived reverberation of the room

Assieme Si riferisce alla capacità di sentirsi l’un l’altro e di suonare insieme

Ensemble Refers to the ease of hearing each other and of playing together

Dinamica Si riferisce alla percezione dei ff e i pp, dei crescendo e dei diminuendo

Dynamics Refers to the perception of the ff and pp, of the crescendo and diminuendo

Avvolgimento del suono Si riferisce alla sensazione di avvolgimento data dal suono

Sound envelopment Refers to the sensation of envelopment given by the sound

Intensità Si riferisce all’intensità del suono

Sound strength Refers to the perceived loudness

Tempo Si riferisce alla facilità di tenere il tempo

Tempo Refers to the ability to keep the tempo

Percezione di se stessi Si riferisce alla facilità di sentire il proprio strumento rispetto all’orchestra

Own instrument perception Refers to the ease of perceiving one’s own instrument among the rest of the orchestra

Bilanciamento tonale Si riferisce all’equilibro tra i suoni gravi e quelli acuti

Tonal balance Refers to the balance between high and low frequencies

Timbro Si riferisce all’influenza dell’ambiente sul timbro del suono

Timbre Refers to the influence of the room on the sound timbre

The complete questionnaire is given in Table III in Italian and in English, with the exact wording of the questions and the instructions that were given to the participants.

4. Analysis methods Subjective data ascribed to ordinal scales, where different items are put in a defined order as in the present survey, should not be analyzed with usual statistical methods [9]. Different types of measurement scales involve different properties and ordinal scales do not allow certain operations, such as addition, to be made. Therefore mathematical elaborations, such as average and standard deviation, are formally forbidden. Specific analysis methods that are suitable for ordinal scales have been developed, for instance the OWA (Ordered Weighted Average)

method, which is used for averaging [10] and the Condorcet method which is useful for ranking [11]. The Condorcet method is named after Marie Jean Antoine Nicolas de Caritat, Marquis de Condorcet (1743– 1794) and was originally developed at the French Academy of Sciences for electoral purposes. Votes are counted by pitting each candidate against all the others in a series of imaginary one-to-one contests. When all possible pairings of candidates have been made, if one candidate beats all the others he/she is declared the “Condorcet winner”. In order to find the overall ranking, it is necessary to reiterate the algorithm taking away the winner of the previous iteration. Resolution is very rough, as each pairing only sets the winner or non-winner. In order to obtain a better resolution, a modified Condorcet method has been proposed [12], but this has led to the inconvenience of increasing its intrinsic complication even more.

1063


ACUSTICA


Table IV. Comparison of the overall rankings obtained with the Borda count (B), the Condorcet method (C) and the modified Condorcet method (M), applied to the scores of the Rai National Orchestra, playing in the Agnelli Auditorium. The following abbreviations are used: “Clarity” (CLA), “Reverberance” (REV), “Ensemble” (ENS), “Dynamics” (DYN), “Sound Envelopment” (SENV), “Sound Strength” (SS), “Tempo” (TMP), “Own Instrument Perception” (OIP), “Tonal Balance” (TB) and “Timbre” (TBR). CLA

REV

ENS

DYN

SENV

SS

TMP

OIP

TB

TBR

B

m sm Ranking

0.72 0.04 I

0.49 0.04 V

0.64 0.05 II

0.51 0.04 IV

0.37 0.04 X

0.42 0.04 VII

0.38 0.04 IX

0.56 0.04 III

0.44 0.04 VII

0.47 0.05 VI

C

m sm Ranking

0.93 0.13 I

0.49 0.15 IV

0.82 0.20 II

0.36 0.07 VII

0.36 0.07 VII

0.38 0.09 V

0.36 0.07 VII

0.56 0.20 III

0.36 0.07 VII

0.37 0.09 VI

M

m sm Ranking

0.78 0.05 I

0.51 0.06 IV

0.67 0.06 II

0.51 0.06 IV

0.34 0.05 IX

0.40 0.05 VIII

0.34 0.05 IX

0.57 0.06 III

0.41 0.05 VII

0.46 0.06 VI

Moreover, there is a possibility that “Condorcet’s paradox” could occur when candidate A results to be the winner against candidate B, B is the winner against candidate C, but C is the winner against A, and the “Condorcet winner” is not identified. The Condorcet method can even lead to contradictions: the principle of pair choice can in fact provide distortions because the winner can depend on the pairing order. For these reasons, the Borda count [13] was preferred by the French Science Academy to the Condorcet method. The Borda count, named after Jean-Charles de Borda (1733–1799), transforms the ranking provided by each voter, given n candidates to choose from, into a numerical representation assigning n points to the candidate placed first, n − 1 to the second and so on, down to one point for the candidate placed last. The overall ranking is obtained adding up the points given to each candidate. The Borda count creates the least number of paradoxes, but it is vulnerable to strategic voting [14], i.e. as the results depend on the sum of the points given to each candidate, a voter can give no points to all the others, in order to produce an advantage for one particular candidate. To avoid this inconvenience, the present authors have assigned an equal value to each missing score so that the sum of the scores given by each voter remains constant. The Borda count, which is easier to apply and offers a better resolution, was proposed in this work as an alternative to the Condorcet method, even though it is formally not suitable for ordinal scales as it is based on addition. In practice, in the case of subjective evaluations ascribed to ordinal scales, the variation of consecutive interval amplitudes is frequently not excessive, so that simpler methods applying average and standard deviation, such as the Borda count, yield meaningful results [12]. If simpler methods are used they need to be validated by means of a comparison with more rigorous methods. For this comparison the concept of uncertainty should be considered. At present, there are no generally accepted procedures for uncertainty assessment in the context of subjective evaluations. The main standard for uncertainty calculation

1064

is ISO GUM [15], which defines “uncertainty” in clause 2.2.3 and the more specific term “expanded uncertainty” in clause 2.3.5, but does not take into account subjective data. However, the ISO GUM has been considered as the reference standard for the present uncertainty evaluation. The standard deviations sm of the mean values were calculated to assess the uncertainty associated with the Borda count. Instead, in the case of the Condorcet and modified Condorcet methods, as they are not described by a mathematical formula, it is not possible to analytically estimate the variability of the ranking results. In order to overcome this drawback, the statistical bootstrap method [16, 17] was applied. This method, generally applied to assess the variability of statistical estimators, introduces the notion of bootstrap sample: N being the number of data of the original experimental sample, a bootstrap sample is obtained by N extractions with replacement of the data contained in the original sample.

5. Results 5.1. Results of the Rai National Orchestra in the Agnelli Auditorium The Borda count, the Condorcet method and the modified Condorcet method were applied to the scores from the Rai Orchestra, after playing in the Agnelli Auditorium, as a preliminary test in order to find the most effective methodology for the whole data set. Table IV shows the mean scores m, obtained after the normalization of the data in the 0–1 interval to make the results of the different methods numerically comparable, the standard deviations of the mean values sm and the overall rankings. In order to assess the standard deviation associated with the Condorcet method and the modified Concorcet method, the previously described bootstrap method was used, and 100 bootstrap sample extractions were performed, as suggested by Efron [17]. It is possible to observe that all three methods assign the first three positions to “Clarity” (CLA), “Ensemble”


(ENS) and “Own Instrument Perception” (OIP), respectively. The Borda count and modified Condorcet method give aligned overall rankings, while the Condorcet method is not able to resolve positions below VI. In the case of the Condorcet method, each result has a resolution of 1 over the number of voters n, while the resolution for the Borda count and the modified Condorcet method is better. Figure 1 compares the results obtained using the three methods and indicates the uncertainty bars of the mean values. The null hypothesis of normal distribution is generally not rejected, therefore the uncertainty value was calculated at a confidence level of 95% using the relevant Student-t distribution. A visual inspection of Figure 1 shows, as expected, wider uncertainty bars for the Condorcet method and similar and narrower ones for the other two methods, which offer a better resolution. The simpler Borda count, which has been validated through its compatibility with the other two methods, was chosen for the subsequent analyses. This compatibility [18] can be evaluated either by means of an approximate method, from a visual inspection of the overlapping of the uncertainty bars, or through a more rigorous method, calculating the Normalized Error (see section 5.4). In Figure 1, considering the Borda count results, it is possible to notice groups of compatible attributes with overlapped uncertainty bars for which a real difference cannot be set. The musicians gave similar scores to the acoustical attributes relative to the same group. A first group containing “Clarity” (CLA) and “Ensemble” (ENS), a second group containing “Reverberance” (REV), “Dynamics” (DYN) and “Own Instrument Perception” (OIP), and a third group including “Sound Envelopment” (SENV), “Sound Strength” (SS), “Tempo” (TMP), “Tonal Balance” (TB), and “Timbre” (TBR), can be identified. “Own Instrument Perception” instead does not seem to be attributable to any single group, as it is compatible with more than one group. In a case like this a more accurate analysis could be useful. If the results of the Condorcet method are examined, it is possible to see that it is even difficult to obtain a ranking among acoustical attributes with very different scores, such as CLA and REV. 5.2. Comparison between the RNO-AA and TROTR Comparisons of the different orchestra-auditorium configurations allow the possible influence of the space on the musicians’ ranking to be investigated. The first comparison involves one of the configurations for which the acoustical conditions of the performance spaces were very different, with the additional advantage of the availability of the complete set of ten attributes. Figure 2 shows the comparison of the priority classification with the uncertainty bars obtained with the Borda count for the Rai National Orchestra, which usually plays on the Agnelli Auditorium stage, and the Teatro Regio Orchestra, a resident orchestra that performs in the Teatro Regio opera pit. The two pairs were abbreviated with the acronyms RNO-AA and TRO-TR, respectively.


Vol. 95 (2009)

Figure 1. Comparison of the normalized scores with uncertainty bars obtained using the Borda count, the Condorcet method and the modified Condorcet method for the Rai National Orchestra in the Agnelli Auditorium. See the caption in Table IV for the used abbreviations.

Figure 2. Comparison of the normalized scores with uncertainty bars obtained using the Borda count for the Rai National Orchestra in the Agnelli Auditorium (RNO-AA) and for the Teatro Regio Orchestra in the Teatro Regio of Turin (TRO-TR). The former is an orchestra-concert hall configuration, the latter an orchestraopera house configuration. See the caption in Table IV for the used abbreviations.

It is clear from a visual inspection of Figure 2 that CLA, DYN, SENV, TB, TBR, SS and OIP show compatible results, while REV, ENS and TMP are not compatible. CLA and ENS are the most important attributes, even though ENS is much more important for TRO-TR than for RNOAA. REV and TMP show different degrees of importance for the two Orchestras: REV is much more important for RNO-AA, while it is the least important aspect for TROTR, and TMP is much more important for TRO-TR, and is at the same level as CLA and ENS. 5.3. Comparison between the RNO-AA and RNORA The second comparison of the orchestra-auditorium configurations involves the Rai National Symphony Orchestra, which took part in the appraisal after having played in the G. Agnelli Auditorium and the Rai Auditorium. In this case, and for all the other orchestra-auditorium pairs, the analyses were based on nine acoustical attributes instead of ten because, as explained in section 3, “Ensemble” has been excluded from the survey.

1065


ACUSTICA

Figure 3. Comparison of the normalized scores with uncertainty bars obtained using the Borda count for the Rai National Orchestra in the Agnelli Auditorium (RNO-AA) and for the Rai National Orchestra in the Rai Auditorium (RNO-RA). See the caption in Table IV for the used abbreviations.


many years, and therefore possibly assigned a higher score to CLA than was expected. A possible “renewed-stage effect” could have been present in the answers and this could have led to give the highest score to CLA, which is in the upper part of the ranking for all the orchestras, as shown below. Another explanation for the higher CLA score could be the connection between clarity and sound envelopment, as mentioned by the musicians in the informal interviews (see section 3). As far as the higher score for sound envelopment is concerned, the reason could be due to the fact that Rai Auditorium stage is smaller and has a shape which gives or inspires a greater sense of sound envelopment with respect to the larger one in the Agnelli Auditorium. In fact, the former is delimited by a 1.9 m high rear concave wall which constitutes a balustrade for the chorus area behind, while the latter is rather open with lateral walls far from the area occupied by the musicians. 5.4. Comparison of all the case studies

Figure 4. Comparison of the normalized scores with uncertainty bars obtained using the Borda count for all the orchestra- auditorium configurations. See the captions in Tables IV and V for the meaning of the abbreviations and acronyms.

Figure 3 shows the comparison between the priority classifications obtained with the Borda count for the two aforementioned configurations. From a visual inspection of the uncertainty bars, all the attributes show compatible results except for CLA and SENV, which are borderline. A hypothetical memory effect that could influence the scores is questionable because the number of the musicians was not exactly the same and the musicians were different in the two auditoria. Furthermore, the two surveys were carried out about six months apart. A complementary research, aimed at collecting musicians’ judgment in the two auditoria and which was based on the same acoustical attributes [19], shows better evaluations in the Rai Auditorium than in the Agnelli Auditorium for most of the attributes, thus excluding the influence of the acoustical perception of the hall on the priority classification. Moreover, the slightly higher values in the Agnelli Auditorium are also related to the objective parameters measured on the stage and reported in Table II. On the other hand, in order to justify the higher score for CLA and SENV in the Rai Auditorium, a possible “stage effect” could be considered. In the informal interviews, the musicians were enthusiastic about the recently restored Rai Auditorium, where they had come back to play after

1066

Table V lists the final classification of the nine acoustical attributes based on the average score values obtained from the application of the Borda count for the six orchestraauditorium configurations. The acoustical attribute “Clarity” is the most important in four cases out of six and “Tempo” is the most important in the other two. However, “Clarity” is in the upper part of the ranking for all the orchestras as is “Tempo”, except for RNO-RA and RNO-AA where it appears in the lower part. “Own Instrument Perception” and “Dynamics” are in the middle of the ranking. “Timbre” and “Sound Envelopment” alternatively score low and intermediate results, and they only scored second and third for RNO-RA. “Tonal Balance” and “Sound Strength” are in the middle-low positions and “Reverberance”, surprisingly, is the least important attribute in the musicians’ priority in four cases out of six. It only scored fourth for RNO in AA and seventh for RNO in RA. The final results, without taking into account the information of uncertainty, could however lead to misleading conclusions. Figure 4 shows a comparison between the mean scores and the related uncertainty bars. The classifications in Table V seem to be very different, whereas the representation of the results with the associated confidence intervals shows an overlapping of most of the uncertainty bars. From an observation of Figure 4, the following main conclusions can be drawn: • the TPO-VC scores have the highest uncertainty, probably because of the limited number of musicians; • all the groups seem to show compatible results for each acoustical attribute, with the exception of “Tempo” for RNO-AA and RNO-RA, which has been judged less important; • “Clarity” seems to be the most important attribute. The compatibility of the results for the same acoustical attribute from the different orchestra-auditorium configurations shows how the space where the orchestra usually



Vol. 95 (2009)

Table V. Priority classifications based on the overall ranking obtained with the Borda count for the six orchestra-auditorium configurations. The following acronyms are used: RNO-AA for the Rai National Orchestra in the Agnelli Auditorium, RNO-RA for the Rai National Orchestra in the Rai Auditorium, TPO-VC for the Turin Philharmonic Orchestra in the Verdi Conservatory Concert Hall, VCO-VC for the Verdi Conservatory Orchestra in the Verdi Conservatory Concert Hall of Turin, SSO-ST for the Sanremo Symphony Orchestra in the Sanremo Casino Theatre, TRO-TR for the Teatro Regio Orchestra in the Teatro Regio of Turin. Ranking

RNO-AA

RNO-RA

TPO-VC

VCO-VC

SSO-ST

TRO-TR

I II III IV V VI VII VIII IX

CLA OIP DYN REV TBR TB SS TMP SENV

CLA TBR SENV OIP TB DYN REV SS TMP

TMP DYN SENV CLA OIP SS TBR TB REV

CLA TMP DYN SENV OIP TB SS TBR REV

TMP CLA OIP DYN TB TBR SS SENV REV

CLA TMP DYN TBR OIP TB SENV SS REV

play, even though characterized by different acoustical features, as shown in Table II, does not influence the priority classification. The above considerations, which were only made from an observation of the uncertainty bars in Figure 4, are quite rough, therefore more rigorous results were needed. In order to make a better comparison of the six orchestraauditorium configurations, the Normalized Error concept [20] was adopted. This is useful for comparisons of measurement results produced at the same hierarchical level, i.e. where no value can be taken as the reference value. In this case, it is necessary to understand whether the difference in the compared results is due to an effective difference between the evaluated properties or to a significant bias between the assessment tools, rather than to random effects. The Normalized Error is calculated as the ratio between the absolute value of the difference of two states of the evaluated property and the relevant expanded uncertainty [15]. Considering a state of the evaluated property, named “1”, which is defined by the average value m1 and its standard deviation sm1 , and a state named “2”, defined similarly by m2 and sm2 , the expanded uncertainty U of the difference m2 − m1 is given by U = tα,ν s = tα,ν

s2m1 + s2m2 ,

(1)

where tα,ν is the Student t-distribution value for an acceptable risk of error α and a number of degrees of freedom ν. The Normalized Error EN is hence calculated using the formula EN =

|m1 − m2 | = U

|m1 − m2 | tα,ν

s2m1 + s2m2

.

(2)

In the presented application, state “1” is the configuration under consideration for each acoustical attribute and state “2” is the overall average of all the configurations. This analysis can be considered a particular kind of hypothesis test. If the EN value is higher than unity, the difference between the two values, m1 and m2 , is higher than its uncertainty, therefore the difference is not merely due

to random effects and the two results can be considered incompatible. On the contrary, if EN is lower than unity, the difference could be due to random effects and the two results can be considered compatible. Values lower than unity do not mean that real differences or systematic effects are not present, but that random effects cover their presence. Table VI shows the means of all the nine acoustical attributes with the associated standard deviations for the six orchestra–auditorium pairs together with the Normalized Errors relative to their overall average mm . The bold numbers represent higher EN values than unity, i.e. the incompatible values with respect to the overall average. Some incompatibilities can be detected for RNO-AA and RNORA. In particular, RNO-AA is incompatible for “Reverberance”, RNO-RA for “Clarity” and “Timbre” and both RNO-AA and RNO-RA for “Tempo”. If the relevant mean values are compared with the overall average mean values, it is evident that “Clarity”, “Timbre” and “Reverberance” have significantly higher scores than the average, while “Tempo” has a lower score. Some of these incompatibilities can be explained, even though some other open questions remain. For instance, looking at the incompatibility of the “Tempo” attribute (TMP), which scored lower than the overall average for AA and RA, both of which were assessed by the RNO, a possible “orchestra effect” could be suggested. As far as the incompatibility of the “Clarity” attribute (CLA) is concerned, which scored higher than the overall average for RA, the “stage effect” described in section 5.3 can be presumed. In order to obtain the final overall ranking with the relative uncertainty, an analysis of the systematic effects was performed referring to the Normalized Errors. The scores with higher EN values than unity (shown in bold in Table VI), except for “Timbre” where the EN value is very close to unity, were not considered, as the averaging of the systematically different values is not correct [20]. Figure 5 shows the final ranking of the nine acoustical attributes. In order to highlight the presence of differences between the attributes, a 20% risk of error of the first kind

1067


ACUSTICA


Table VI. Overall ranking obtained with the Borda count for the six orchestra-auditorium configurations with the Normalized Errors, EN , relative to their overall average, mm . The bold numbers represent higher EN values than unity, i.e. the incompatible values with respect to the overall average. CLA

REV

DYN

SENV

SS

TMP

OIP

TB

TBR

RNO-AA

m s EN

0.74 0.04 0.1

0.51 0.04 1.5

0.53 0.04 0.0

0.39 0.04 0.8

0.43 0.04 0.2

0.39 0.04 1.5

0.58 0.04 0.4

0.45 0.04 0.1

0.48 0.05 0.1

RNO-RA

m s EN

0.86 0.03 1.4

0.40 0.06 0.2

0.43 0.06 0.8

0.57 0.06 0.7

0.38 0.07 0.2

0.32 0.05 1.9

0.51 0.06 0.2

0.43 0.07 0.0

0.59 0.05 1.0

TPO-VC

m s EN

0.56 0.13 0.6

0.34 0.08 0.1

0.56 0.10 0.2

0.56 0.10 0.4

0.52 0.09 0.6

0.60 0.13 0.2

0.55 0.09 0.1

0.37 0.10 0.3

0.44 0.11 0.1

VCO-VC

m s EN

0.85 0.06 0.8

0.31 0.07 0.4

0.56 0.05 0.3

0.51 0.08 0.2

0.35 0.06 0.5

0.63 0.09 0.4

0.50 0.08 0.2

0.44 0.08 0.0

0.33 0.08 0.8

SSO-ST

m s EN

0.68 0.08 0.3

0.33 0.07 0.2

0.52 0.06 0.1

0.37 0.08 0.6

0.41 0.05 0.1

0.69 0.07 0.9

0.58 0.06 0.3

0.49 0.07 0.4

0.44 0.07 0.1

TRO-TR

m s EN

0.72 0.05 0.1

0.30 0.04 0.6

0.56 0.04 0.3

0.42 0.05 0.4

0.39 0.04 0.2

0.65 0.05 0.8

0.49 0.05 0.4

0.46 0.04 0.2

0.50 0.05 0.3

Average

mm sm

0.74 0.03

0.37 0.03

0.53 0.03

0.47 0.03

0.41 0.02

0.55 0.03

0.54 0.03

0.44 0.03

0.46 0.03

Table VII. Priority groups of acoustical attributes assuming a 20% risk of error. Acoustical attribute

Figure 5. Final overall ranking of all the acoustical attributes with the relevant uncertainty bars. The results were obtained excluding the systematically different values and assuming a risk of error of 20%. See the caption in Table IV for the used abbreviations.

(the test shows a difference only due to random effects) was assumed. Consequently, the risk of the second kind (a real difference is present but not shown in the test) was reduced. This leads to a decrease in the uncertainty bars. In this way, a better visualization of the priority classification of the acoustical attributes is obtained and the four clearly separated priority groups reported in Table VII are identified. These four groups, sorted in order of importance from 1 to 4, are composed of acoustical attributes with compatible scores. The most important attributes, “Clarity” and “Tempo”, are in group 1. “Own Instrument Perception” and “Dynamics” belong to group 2, while group 3 contains “Sound

1068

Clarity Tempo Own Instrument Perception Dynamics Sound Envelopment Timbre Tonal Balance Sound Strength Reverberance

Priority group 1 1 2 2 3 3 3 3 4

Envelopment”, “Timbre”, Tonal Balance” and “Sound Strength”. “Reverberance” is in group 4, and represents the least important attribute.

6. Discussion Some considerations can be drawn on the final classification of the acoustical attributes. Higher scores are given to “Clarity”, “Tempo” and “Ensemble”, which can be related to ensemble playing [3, 4, 21], while “Own Instrument Perception” and “Dynamics” can be connected to the playing technique. According to Meyer [3], there is in fact a close interaction between the orchestral sound heard by the individual musicians and their playing technique, and the limits of the dynamic range can depend on the playing technique. Moreover, Naylor and Craik [2]



state that a musician must be able to hear himself, so that he/she can make adequate judgments of his/her own pitch, timbre, dynamics and phrasing. “Sound Envelopment”, “Timbre”, “Tonal Balance” and “Sound Strength”, together with “Reverberance”, which is considered the least important attribute from the musicians’ point of view, can be associated with the overall acoustic response of the performance space. These attributes do not directly concern playing quality, but can however influence the performance [1, 3, 5]. As stated by Naylor “. . . performers . . . want to hear fine sound quality, at the same time they must be satisfied with informational aspects of their acoustical environment, to allow them to produce a correct performance.” [21]. As far as the perceived “Reverberance” is concerned, many authors have reported the limited influence of reverberation on musicians’ performance, even though referring only to small groups. As stated by Gade [4], low levels of reverberant sound are generally preferred for ensembles and this may be due to the masking effect of reverberation. An unmasked direct sound is desirable. Marshall and Meyer [22] reported that, for small groups, reverberation is not important for an ensemble. Ueno et al. [6] found that, for chamber music performance, the ease of “making harmony” and the ease of “hearing each other” are related to the energy of reverberant sound rather than the reverberation time for musicians. A further outcome of the research is the incompatibility between “Clarity” and “Reverberance”, as highlighted in Figure 4 and in Table VII, where “Clarity” is in the first priority group while “Reverberance” is in the last one. With respect to this finding the following considerations can be made: 1. The incompatibility in the priority classification does not necessary imply the absence of correlation between the two attributes, even though the absence of correlation between “Clarity” and “Reverberance” is mentioned in works concerning listeners’ assessment in auditoria [23, 24, 25]. 2. It is not possible to exclude the influence of the same physical characteristics on the two attributes. Harkness [26] suggested that the “period of existence of sound” (see [3]) is useful to describe the perceived clarity and reverberance. According to him, “Clarity” can be assessed when playing in the lowest dynamic range, ppp, results to be perfectly clear, while “Reverberance” can be assessed when playing in the highest dynamic ranges, such as fff. Consequently, these attributes are influenced by both the background noise level and the reverberation time, which can make the sound more or less persistent.

7. Conclusions The most important acoustical attributes for musicians playing on a stage or in an orchestra pit have been determined by means of a data ranking analysis. The chosen analysis method was the Borda count which, even

Vol. 95 (2009)

though not suggested for ordinal scales, has been validated through a comparison with the more rigorous and suitable Condorcet method and a modified version of this method. Five professional orchestras participated in the survey which was carried out by submitting questionnaires to musicians after rehearsals on a stage or in a pit. Five orchestra-concert hall configurations and one orchestraopera house configuration were tested. The uncertainty assessment was taken into account in the subjective analyses and both a visual inspection of the overlapping of the uncertainty bars and the more rigorous Normalized Error method were used to examine the compatibility of the results. The latter was applied, in particular, to find compatible results with respect to the overall average. Some incompatibilities can be detected for the Rai National Orchestra in the Rai Auditorium and in the Agnelli Auditorium, while compatible results with the overall average can be observed for the other orchestras. The factors that influence the ranking have been investigated and, apart from the musicians’ background which is difficult to assess, other causes, such as the different acoustical conditions (measured or perceived) in the auditorium do not seem to have affected the results. Only the Rai National Orchestra results suggest the existence of possible orchestra and stage effects or other unknown systematic effects. However, the compatibility of the other orchestras’ judgments shows how the space, even though characterized by different features, does not influence the priority classification. After the exclusion of the systematically different values of the Rai National Orchestra, an overall compatibility analysis showed that the most important acoustical attributes for musicians are “Clarity” and “Tempo”, which can be associated with the accuracy of ensemble playing, followed by “Own Instrument Perception” and “Dynamics”, which are more related to the playing technique. “Sound Envelopment”, “Timbre”, Tonal Balance” and “Sound Strength” form a third group of importance while “Reverberance” represents the least important attribute. From the comparison between the Rai National Orchestra in the Agnelli Auditorium and the Teatro Regio Orchestra in the Teatro Regio, which involved ten instead of nine acoustical attributes including “Ensemble”, it has emerged that “Ensemble” was placed in the first group of importance together with “Clarity”. In conclusion, if only the homogeneous group of answers is considered, musicians seem to judge attributes that deal with the accuracy of ensemble playing and the playing technique to be more important than attributes related to the acoustics of the performance space, even though the quality of the playing aspects depends on the acoustics. References [1] A. H. Marshall: Acoustical conditions preferred for ensemble. J. Acoust. Soc. Am. 64 (1978) 1437–1442.

1069


ACUSTICA

[2] G. M. Naylor, R. J. M. Craik: The effect of level difference and musical texture on ease of ensemble. Acustica 65 (1988) 95–100. [3] J. Meyer: The sound of the orchestra. J. Audio Eng. Soc. 41 (1993) 203–213. [4] A. C. Gade: Investigations of musicians’ room acoustic conditions in concert halls. I: Method and laboratory experiments. Acustica 69 (1989) 193–203. [5] A. C. Gade: Investigations of musicians’ room acoustic conditions in concert halls. II: Field experiments and synthesis of results. Acustica 69 (1989) 249–262. [6] K. Ueno, T. Kanamori, H. Tachibana: Experimental study on stage acoustics for ensemble performance in chamber music. Acoust. Sci. & Tech. 26 (2005) 345–352. [7] J. Sanders: Suitability of New Zealand Halls for chamber music. Auckland, Marshall Day Acoustics Pty Ltd, 2003. [8] K. Ueno, H. Tachibana: Cognitive modelling of musician’s perception in concert halls. Acoust. Sci. & Tech. 26 (2005) 156–161. [9] S. S. Stevens: On the theory of scales and measurement. Science 103 (1946) 677–680. [10] F. Franceschini, M. Galetto, M. Varetto: Qualitative ordinal scales: the concept of ordinal range. Qual. Eng. 16 (2004) 515–524.


[14] P. Dasgupta, E. Maskin: The fairest vote of all. Sci. Am. 290 (2004) 92–97. [15] ISO Guide 98: Guide to the expression of uncertainty in measurement (GUM). International Organization for Standardization, Genève, 1995. [16] B. Efron: Bootstrap methods: Another look at the Jackknife. Ann. Statist. 7 (1979) 1–26. [17] B. Efron, R. J. Tibshirani: An introduction to the Bootstrap. Ed. Chapman and Hall, New York, 1993. [18] JCGM 200: International vocabulary of metrology. Basic and general concepts and associated terms. VIM, 3rd edition, BIPM, Sévres, 2008. [19] M. Giovannini, A. Astolfi: The subjective assessment of musicians in spaces for music performance. Proceedings of the 7th International Conference on Acoustics, Oslo, 2008. [20] ISO/IEC Guide 43-1: Proficiency testing by interlaboratory comparisons. Part 1: Development and operation of proficiency testing schemes. International Organization for Standardization, Genève, 1997. [21] G. M. Naylor: The achievement of ensemble. Appl. Acoust. 23 (1988) 109–120. [22] A. H. Marshall, J. Meyer: The directivity and auditory impressions of singers. Acustica 58 (1985) 130–140. [23] R. J. Hawkes, H. Douglas: Subjective acoustic experience in concert auditoria. Acustica 24 (1970) 235–250.

[11] M. Condorcet: Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix. 1785. [12] G. Barbato, S. Farné, G. Genta: Management of subjective evaluations represented by ordinal scales. Proceedings of the 12th IMEKO TC1 & TC7 Joint Symposium on Man Science and Measurement, Annecy, 2008.

[25] M. Barron: Subjective study of British symphony concert halls. Acustica 66 (1988) 1–14.

[13] J. C. Borda: Mémoire sur les élections au scrutin. Histoire de l’Académie Royale des Sciences, 1781.

[26] E. L. Harkness: Performer tuning of stage acoustics. Appl. Acoust. 17 (1984) 85–97.

1070

[24] M. Barron: The Gulbenkian Great Hall, Lisbon. II: An acoustic study of a concert hall with variable stage. J. Sound Vibration 59 (1978) 481–502.