It has been noted that the presence of saliva in the pharynx and larynx during flexible endoscopic evaluation of swallowing (FEES) can be an indicator of increased risk of aspiration and consequent pneumonia, as well as weight loss and malnutrition. However, to date, little is known about what characteristics of these secretions are associated with these risks. This article describes the development of a standardised, clearly defined scale that can support the assessment of the characteristics called the New Zealand Secretion Scale (NZSS). Development of a new tool requires multiple steps and the authors describe the model they used to guide them in this. They outline the development of the scale based on a comprehensive review of the literature. The NZSS comprises three observable parameters including location, amount and response to accumulated secretions. The tool was reviewed by an expert panel before pilot testing commenced to evaluate and refine scale properties. Two researchers reviewed 187 endoscopy videos using the NZSS during the pilot, resulting in an adjustment of the scoring criteria and categories. This study focused on validating the content and psychometric properties of the tool, using 254 endoscopy videos. Ten video samples were rated by both researchers, experts in performing FEES and a convenience group of 28 untrained speech and language therapists. Results demonstrated the NZSS has desirable measurement properties, internal consistency and intra- and inter-rate reliability. This type of rigorously developed measure is extremely valuable in the field of speech and language therapy. It provides a useful and well-designed tool for clinicians to quickly use and interpret in a busy clinical setting.