Research Interests

Papers

See my Google Scholar list of papers.
Go to a chronological list of papers.

Outline

My primary interest is in design principles and practical methods and tools for designing effective information systems, like web sites.

To be effective, information access systems have to comply to a lot of criteria, including: usability, accessibility, security; they have also to evoke the desired emotions and phsychological effects. It is no wonder that many web sites fail to meet high standards along these lines. Some of the projects listed below aim at sheding some light on these aspects.

The book [crearesitiweb] focuses on basic principles underlying design and analysis of web sites and applications.

User experience

UX evaluation

I recently worked on the influence that online advertising has on usability aspects of newspapers. We found quite a rich and non-homogeneous response by the 70+ participants we involved in the study.
With my students we produced and finished evaluating (Feb 2011) a questionnaire to measure the emotional response elicited by web pages.
See also a review of online advertising effects on usability.

Interaction design

I started (Dic 2010) working on interaction design, and specifically on a framework based on the UML for the rapid prototyping of user interfaces to support usability investigations. Basic ideas are discussed in [fsmbui2010].

Usability of text-only Pages

Text-only pages are normally considered as second-level pages for second-level web surfers. However well-implemented text-only pages might be an opportunity rather than a hindrance.

The goals of this research was (1) to determine if and for whom, text-only pages are useful; (2) to understand what aspects of text-only pages make them more attractive and pleasant. In particular the research focussed on determining effectiveness, productivity, satisfaction and emotional effects of text-only pages with respect to users that are disabled or that use PDAs.

Papers: a discussion of pros and con's of text-only pages produced on the fly [41], and experimental evaluations with respect to disabled people and users of PDAs: [43], [47], [49].

Accessibility methods

Validity and reliability of WCAG

A recent research project deals with evaluating quality of web accessibility guidelines. In the study described in [66] we experimentally tested WCAG 1.0 and WCAG 2.0 with respect to validity (the extent to which all and only the true guideline violations are found) and to reliability (the extent to which independent evaluators produce the same results). The test, performed with 35 junior web developers minimally experienced with accessibility, suggests that

  1. none of checkpoints reach the 80% threshold, claimed by WAI to be minimum agreement between evaluators in order to call "testable" a checkpoint;
  2. there are differences between WCAG 1.0 and 2.0; they are generally small and in favor of WCAG 1.0;
  3. our expectations on which checkpoints should be less reliable were not supported by results (for example, "1.4.4 Reside Text" appears to be less reliable than "1.3.1 Info and Relationships").

The file experiment-assets09.zip contains the raw data produced in the experiment and used to compute the results discussed in the paper.

A more recent experiment was performed (data available at experiment-wcag20-2010) concerning validity and reliability of WCAG 2.0, both by expert evaluators and novice ones. Preliminary results were presented at Assets 2010 (see [71]), and a more detailed journal paper is under preparation. Three tables present the 61 success criteria ordered by testability, accuracy rate and difficulty of evaluation on the basis of data collected from experts.

Barrier walkthrough: an accessibility evaluation method

Full details of the barrier walkthrough method are available.

See a study on effectiveness of the barrier walkthrough method for assessing web accessibility.

Web Accessibility Metrics: SAMBA

This work focusses on methods for measuring accessibility levels of highly dynamic and large web sites. SAMBA is a semi-automatic methodology for measuring levels of accessibility, where accessibility is assumed to be related to the effectiveness with which people with disablities can use a website. SAMBA combines output produced by a testing tool with human judgment (on just a sample of the output) to yield an overall index of accessibility that considers severity of the barriers and error rates of tools.

[52] describes preliminary ideas about a practical accessibility metrics, further refined and tested in [56]. [54] describes an experimental comparison between different metrics. [59] describes MAMBO, a procedure for measuring web accessibility that is based only on human assessments of accessibility.
See an experimental evaluation and comparison of several automatic metrics for accessibility metrics2011.

Web Quality Assurance

Usability, accessibility and security are three basic components of the quality of a website. High levels of quality can only be achieved if proper organization, work processes and tools for usability and accessibility are deployed.

The goal of this research is (1) to define key processes that are necessary to achieve high level of quality in website development, (2) to understand the role of automatic tools to be used in such processes, and (3) to determine methods for assessing the level of quality of a website.

Papers: a comparison method for effectiveness of testing tools [40]; a paper describing quality assurance processes [42]; two papers on quality models [39, 38]; a paper on how to validate a quality model [35]; and a survey on web usability testing tools [34]. [57] experimentally compares different sampling methods and their effects on accessibility assessments.

Past

TeQsim is a qualitative simulation project I worked on in the past.