All drawings appearing in this Recommendation have been done in Autocad. Recommendation E.420 CHECKING THE QUALITY OF THE INTERNATIONAL TELEPHONE SERVICE - GENERAL CONSIDERATIONS 1 Quality of service parameters An adequate picture of the level of quality of service (QOS) in the network can be defined by a set of parameters which are measured, registered and data processed. In Recommendation E.800 a set of performance concepts is defined in order to provide a satisfactory description of the quality of service, and the interconnection of those concepts is shown. Each performance concept can be impaired by a number of particular causes. These causes, either singly or in groups, lie behind the failure symptoms observed by the user. A user views the provided service from outside the network and his perception can be described in observed quality of service parameters. The link between the observed quality of service parameters and the impairment causes can be indicated in the form of tables.1) Five main observed quality of service parameters are derived; they reflect the quality of: i) providing the customer with the ability to use the desired services; ii) furnishing a desired level of service for: - connection establishment, - connection retention, - connection quality, - billing integrity. These main parameters can be supervised by quality of service indicators (e.g. efficiency rate, call cut-off rate, etc.). Objectives can be set for these indicators and can be revised at regular intervals. When a deterioration of these supervision indicators is detected, or when an improvement programme is started, more data must be collected by measurements to permit a more detailed analysis in order to locate the impairment causes which lie behind the observed problem areas. 2 Methods of measuring the quality of service 2.1 The following methods of measuring the quality of service are described: 1) service observations by external means; 2) test call (simulated traffic); 3) customer interviews; 4) internal automatic observations. 2.2 Administrations are recommended to draw up a programme for observations and tests designed for assessment of circuits and equipment, supervision of operators and evaluation of the quality of service given to subscribers. It would be desirable if telephone Administrations were to exchange statistics on quality of service. 2.3 Table 1/E.422 relates to the manual and semi-automatic observations of the quality of international automatic and/or semi-automatic service. It provides in particular a check of the percentage of unsuccessful calls due to technical faults (equipment shortages or failures). Table 2/E.422 relates the same information as Table 1/E.422 but does not include information which can only be obtained by operators listening in (automatic observation). Table 1/E.423 relates to observations on traffic set up by operators. It provides, in manual and semi-automatic service, a means of determining the efficiency of international circuits, of assessing the work of operators and the quality of transmission. Table 2/E.423 summarizes observations of the time-to-answer by operators. The table is compiled by automatic means. Table 1/E.424 is used to record the results of test calls undertaken especially when the observations shown in Table 1/E.422 make it clear that the percentage of faults is too high. The use of customer interviews as a method of measuring telephone service quality is the subject of Recommendation E.125 which is particularly concerned with the determination of sources of user difficulty which may arise when making an international automatic telephone call. 1) Such tables can be found in the handbook cited in [1]. Fascicle II.3 - Rec. E.420 PAGE1 Recommendation E.426 contains a general guide to the expected percentage of effective international call attempts. Table 1/E.427 may be used to supplement the information contained in Table 1/E.422 when the observations shown in that table make it clear that the percentage of faults due to customer difficulties is too high or the outcome of the application of Recommendation E.125 demonstrates the need for additional information. Recommendation E.425 describes the data that might be taken from the switching centres with respect to quality of service, and the exchange of that data. 2.4 Paying attention to the quality of service of the incoming traffic stream is of major importance, since the incoming Administration is in a better position to improve the situation. In the past less attention has been paid by several Administrations to the quality of service (QOS) on incoming calls than on outgoing calls. This situation should not persist in the future. Therefore, in addition to the measurement of QOS of the outgoing traffic stream which is described in this series of Recommendations, Administrations are strongly advised to observe the incoming traffic stream with the aim to improve the QOS. 3 Other sources of information on the quality of service The following sources are useful to consider when trying to improve the quality of service: - subscriber complaints (see Annex B); - other Administrations or organizations such as INTELSAT (SPADE reports); - operators contacting maintenance staff for direct action; - operators giving information on QOS: if operator traffic is significant one might consider organizing the flow of this type of information by establishing "trouble codes", e.g. echo, no tone, no answer, etc.; - reports from "national" switching centres: the QOS as experienced by the subscriber does not only depend on the international network and the network of the country of destination but also on the national network of the country of origin; - user organizations/large companies: as large companies have much to gain from an improved QOS they might be willing to cooperate with Administrations; - holding time versus conversation time measurements; - average conversation time; - traffic measurements; - transmission measurements. PAGE2 Fascicle II.3 - Rec. E.420 ANNEX A (to Recommendation E.420) A possible approach to integrate activities measuring the quality of service into an overall problem-investigating process The flowcharts of the resource allocation process and a typical problem identification procedure are given in Figures A-1/E.420 and A-2/E.420. The numbers 1) through 10) in the figures correspond to the processes described below. 1) The exception threshold is set to detect possible isolated destinations. It is up to the individual Administration to set the value. 2) A destination could be regarded as being under isolated condition when the bid frequency is significant enough to show that there is some demand to the destination (e.g. 20 attempts per day) without or nearly without answer. 3) The most practical way to find out whether improvements may be possible is "consultation with other Administrations". 4) Apply, if possible, network management actions, e.g. alternative routing. 5) The destination priority, P, for each destination is calculated as follows: P = BID2) x (TABR - MABR) where BID2) is the number of total bids to the destination during a certain period of time (for example, 1 month); TABR is the target ABR3) (answer bid ratio) performance which is expected as the result of the service improvement activities; MABR is the measured ABR to the destination during the same period with BID. The TABR is set for each destination and can be based on the average historic ABR and should be higher than that value. In order to comply with Recommendation E.426, S 2.2, the TABR to be used in the formula for P given above should not be lower than the MABR experienced one period earlier. 6) In order to comply with S 2.4, it is suggested to consider also the total international incoming traffic stream as one of the elements which require QOS improvements. It should be noted that the procedure can be well applied to domestic destinations, for example, on an area code basis, and can be applied on an incoming route basis. 7) Perform detailed analysis: when possible, monitor circuit group performance and do analyses on a destination code basis. It is essential to be aware of "killer trunks" (though observation of the QOS is not directly intended to discover killer trunks). 8) Discuss possible improvements with counterpart. 9) In Recommendation M.710 (General maintenance organization for the international automatic and semi-automatic service) the basic maintenance elements, their functions and the cooperation between the elements are described. Recommendation M.1230 (Assessment of the performance of the international telephone network) gives guidance on the relationship between service quality observations, network performance assessment and maintenance procedures. It should be noted that the QOS very much depends on the proper operation of maintenance elements and maintenance procedures. Therefore, Administrations faced with QOS problems are strongly advised to be attentive to the maintenance Recommendations contained in Volume IV. 10) If this procedure does not lead to a successful conclusion, then an escalation procedure may be required (see Recommendation M.711). 2) Paid minutes or the revenues can be used. 3) In case ABR cannot be used, ASR (answer seizure ratio) is considered to be an acceptable substitute. Seizures, TASR and MASR are then applicable. Fascicle II.3 - Rec. E.420 PAGE1 Figure A-1/E.420 - T0201270-88 Figure A-2/E.420 - T0201280-88 ANNEX B (to Recommendation E.420) Utilization of customer complaints to improve the quality of service for international traffic It is possible to use subscriber complaints to control processes if the organization of an Administration provides possibilities for centralized collection of these complaints. The collected data can be processed statistically to provide useful indications for the operations and maintenance staff to correct problems and, in so doing, improve the quality of service. Three aspects are relevant in the processing of the data: - the data itself; - the statistical processes; - the analysis of complaints. B.1 Data to be collected The trouble report of an individual customer may be subjective and unqualified since it is usually made by a person, who is not well trained for observation of quality of service. Therefore it will be necessary to make sure that the information about the complaint is as reliable as possible and useful for identifying the possible impairment of the network that caused the complaint. Examples (see also the handbook cited in [1]): - data concerning the subscriber numbers involved (route, destination); - data concerning the observations during the unsuccessful call attempt(s), or the disturbed call; - time of the observation by the customer. B.2 Statistical processes to improve the reliability of the data Reliable data is obtained by statistical processing of large numbers of complaints (e.g. an average value during a certain period of time). In order to achieve this, the following methods are considered to be useful: 1) choose complaints whose possible causes seem to relate to impairments of the network; 2) accumulate complaints for a certain period of time, for example, one month or one week, depending on the number of complaints; 3) calculate the ratio of complaints statistically from accumulated data, for example, complaint-to-completion ratio (CTCR), for the chosen period of time: CTCR = eq \f( Number of complaints, Number of effective call attempts) x 100% It is practical to use the CTCR in combination with one or more classification aspects (see S B.3) such as "per destination". B.3 Analysis of complaints It is necessary to identify the possible impairment of the network causing the complaint and smoothly clear this impairment in order to actually improve the quality of service. To accomplish this, the complaint needs to be processed into data useful to network maintenance organizations in localizing the possible impairment. The following methods are considered useful: 1) classifying complaints by category of failure; 2) classifying complaints by destination, route (or circuit group) and/or area code; 3) time of day analysis. This may be effective in identifying impairments that may not be apparent when looked at on a total day basis; 4) highlighting relative changes or trends in the statistical data. These changes are likely to reflect a change of the network status and are useful indications along with the values themselves. For example, a rapid increase in the statistical value (e.g. the ratio of complaints) may reflect a new impairment of the network. Reference [1] CCITT Manual Quality of service, network management and network maintenance, ITU, Geneva, 1984. PAGE2 Fascicle II.3 - Rec. E.420