UIS-Hunter: Detecting UI Design Smells in

Android Apps

Bo Yang

∗

,Zhenchang Xing

†

,Xin Xia

‡

,Chunyang Chen

Zhejiang University, Hangzhou, China

†

Australian National University, Canberra, Australia

‡

visual design smells that indicate violations of good UI design

guidelines. UI design guidelines constitute design systems for a

vast variety of products, platforms, and services. Following a

design system, developers can avoid common design issues and

pitfalls. However, a design system is often complex, involving

various design dimensions and numerous UI components. Lack

of concerns on GUI visual effect results in little support for

detecting UI design smells that violate the design guidelines

in a complex design system. In this paper, we propose an

automated UI design smell detector named UIS-Hunter (UI

Design guidelines gallery website and a tool website. The gallery

website is a back-end knowledge base that attaches conformance

and violation examples to abstract design guidelines and allows

developers and designers to explore the multi-dimensional space

of a complex design system in a more structured way. As a

front-end application, the tool website takes a UI design as input,

returns a detailed UI design smell report, and marks the violation

regions (if any). Moreover, the tool website presents conformance

and violation examples based on the gallery website.

Demo URL: https://uishunter.net.cn/

tween users and software applications. With a poorly designed

Android GUI, users would feel frustrated and uninstall the

application. We refer to poorly designed UIs as UI design

smells. For example, truncating the full title of the top app

bar in Fig. 1(a) hinders the access of key information (i.e.,

end time of a trip). In Fig. 1(b), the background image makes

illegible foreground buttons/text. In Fig. 1(c), the confirmation

dialog provides only a single action, and cannot be dismissed.

Attaching tabs to bottom navigation in Fig. 1(d) can cause

confusion about what action or tab controls which content.

of Material Design. The tool is able to (i) automatically

process UI screenshots or prototype files to detect UI design

smells and generate reports, (ii) highlight the violated UI

regions and list the material design guidelines that the design

smells violate, and (iii) present conformance and violation UI

design examples to assist understanding. The tool consists of

Material Design guideline gallery and a tool website. The

gallery website is a back-end knowledge base that collects

a set of conformance and violation UIs from real Android

apps for each supported visual guideline. Through the gallery

the violated guidelines, highlights the regions violating these

guidelines, and provides conformance and violation examples

to assist developers understanding.

We evaluate UIS-hunter on a dataset of 60,756 unique UI

screenshots of 9,286 Android apps. UIS-Hunter achieves a

precision of 0.81, a recall of 0.90, and F1-score of 0.87. We

also conduct two user studies to evaluate the utility of UIS-

Hunter. Our studies show that UIS-Hunter can detect UI design

smells more effectively than human checkers, and most of the

UI design smells reported by UIS-Hunter are rated as severe

including input UI design parser, atomic UI information

extractors, and UI design validator.

Material design are well-structured and adopt consistent

writing conventions. To construct a knowledge base of UI

design guidelines, we collect and examine guidelines explicitly

marked as “don’t” and “caution” and illustrated with at least

one UI design example. Then, we identify the information

about primary component, component design aspects, and

design dimensions from document headings. Finally, we read

After validating all UI components in the input UI with

Fig. 2: Approach overview (illustrating the validation of a navigation drawer as an example)

Material Design guidelines gallery is a back-end knowledge

base for UIS-Hunter. Each design guideline is indexed by the

type of primary component involved in the guideline, and

annotates the guideline with multi-dimensional information

such as component design aspect, design dimension, guideline

severity, and needed atomic UI information. The gallery at-

taches conformance and violation examples to abstract design

guidelines and records the violation condition can be defined

as the first-order logic over the relevant types of atomic UI

information. The gallery collects 130 guidelines for 23 types

Fig. 3: The homepage of Material Design Guidelines Gallery

UIS-hunter is implemented as a web application. Users

can upload their UI data in two types: 1) a UI mockup

created in design tools (e.g., Figma) or GUI builders (e.g.,

Automator). The tool shall process the input UI data in the

We conduct two experiments to investigate the efficiency of

technique [18] to examine the minimum number MIN of

the rest of 50,829 app UIs without reported violations. The

estimated accuracy has 0.05 error margin at 95% confidence

level and we sample and examine 3,490 app UIs in total. UIS-

Hunter achieves 0.81, 0.90, and 0.87 for the precision, recall

and F1, respectively. Based on the manually validated detec-

tion results, UIS-Hunter detects 7,497 unique UIs that contain

true-positive design violations for the 45 don’t-guidelines of

10 types of UI components. UIS-Hunter detects all violation

examples in Fig. 1. The results indicate 12.3% of the 60,756

app UIs contains more than one confirmed design violation.

In addition, 16% of 7,497 contain more than one violations.

Among the 9,286 apps, 6,699 (72%) apps have more than two

To further investigate whether developers can effectively de-

tect UI design smells and how ordinary app users consider the

reported UI design smells, we recruit 5 front-end developers

seriously by majority of users. Five guidelines have recall

less than 0.4. The result indicates developers fail to detect

these five guidelines. For example, “an outlined button’s width

shouldn’t be narrower than the button’s text length” needs to

check whether the button is outlined and then compare lengths.

14 guidelines are considered severe like “don’t mix tabs that

contain only text with tabs that contain only icons”. We

also find some controversial guidelines (i.e., close non-severe

versus severe ratings ratios). For example, booking.com’s

Android app does apply icons to some destinations but not

Design guidelines gallery, a gallery presenting a demographic

study results of Material Design guidelines and instantiating

each guideline with conformance and violation UIs examples.

Through the UIS-Hunter tool, developers can upload their

UI designs in two formats (SVG from design tools and

JSON from UI testing framework) and receive a detailed UI

design smell report. Material Design guidelines gallery allows

designers and developers to search and filter the guidelines and

examples users are interested in and reduce the time cost on