The NCCTP recently considered a questionnaire produced by the DTI-funded Avanti project (the questionnaire itself was apparently based on similar ones issued by a couple of major UK contractors as invitations to tender, ITTs) to get basic information about UK extranet vendors, their hosting, and system functionality. One outcome would be the publication of this information on an Avanti website – something likely to cause palpitations among most of the UK vendors.
As vendors (I obviously speak from experience at BIW Technologies), we are often asked to complete similar questionnaires in order to get onto short-lists for particular projects, customers, etc, and I am sure we all share the same gripes. These are my top moans:
- Confidentiality – Much information provided in response to a questionnaire may be commercially sensitive. Vendors will not want some responses to be in the public domain (ie on a website) and viewable by their competitors.
- Information gets ‘out-of-date’ – Particularly as application service providers (ASPs), the software can be updated several times a year. A questionnaire response will often, therefore, be out-of-date within weeks or months. As an example, a few years ago most of the NCCTP members provided some very basic information for CNPlus’s extranet section, but it quickly got out-of-date.
- ‘Tick in the box’ syndrome – asked if a system has a particular functionality, most vendors will answer ‘yes’ whether their system genuinely offers that functionality “out-of-the-box” or not. For example, it may be achieved through a ‘work-around’, it might require consultancy input (a little or a lot?), or it may warrant some customization or additional development that might be undertaken if the vendor wins the job!
- Evaluating qualitative answers – some topics (security features, infrastructure, etc) may require long and detailed technical responses, and many AEC project teams are ill-prepared to understand the differences.
- Expense involved in responding – ITTs and similar questionnaires can take many man-hours to complete (no two questionnaires are ever the same!), and the volume of additional information (eg: customer lists, case studies, etc) can be substantial – with no guarantee that it will ever be read, let alone properly evaluated as part of the selection process.
- Poorly drafted, imprecise or ambiguous questions – For example, when looking at a vendor’s financial background, should responses relate solely to the extranet service or will the vendor ‘gild the lily’ be reporting on the wider interests of its parent company?
- Inappropriate or irrelevant questions – For example, no point in asking about folder structures if the vendor’s system architecture is based on a relational database.
- My gut feeling is that in the end, outside a few more enlightened organisations, it all boils down to price. In a low-margin industry, the decision can end up being made on the basis of the cost of the service, regardless of the credentials of the vendor, the quality of the hosting environment, functionality, support, etc. Accordingly, we try to get prospective customers and end-users to view a detailed presentation of the software, talk to existing customers and end-users, and try out the functionality for themselves.
1 ping
[…] « The perils of vendor questionnaires […]