Integration of Design Knowledge and Process
A study of the cognitive activity of the designers from the
CENTRO RICERCHE FIAT (CRF) and STILE BERTONE (SB) enabled to investigate
designers’ information process and extract the following essential data:
- Designers sectors
of influence,
- Design cognitive structures and processes (emotional / affective processes),
- Domain specific knowledge (semantic adjectives, Kansei information),
The sectors of influence were used for the database
elaboration and appear also on the GUI. The way the designers search for
information was formalised and transferred into procedures. The latter gave
birth to specific trends functionalities like the TRENDS search capabilities going
from open (with serendipity) to focused. The design cognitive structures and
the domain specific knowledge were integrated into pertimmizers. Moreover, TRENDS
system integrates flexible content-based image retrieval facilities that
utilise ontological referencing, and software able to realise main procedures
relating to Conjoint Trends Analysis. This method which was developed by SERAM enables
the identification of design trends through the investigation of sectors of
influence. The main functionalities are the trends identification and the
pallets generation.
Figure 3: From Conjoint Trends Analysis to TRENDS software
Image description technology
Work related to image content indexing, i.e. the choice of
image descriptors, has been finalized. The relevant software modules (image
retrieval server and the indexing module) have been implemented and delivered
for Prototype 2. The query module also implements a state of the art database
exploration paradigm based on relevance feedback (see figure 3). The system has
been tested with real users and their feedback integrated into the system. In
parallel, the image part continued with work on the construction of pallets and
clustering of images. These results will be used for image categorization to
generate summaries of classes of images and for the definition of harmonies of
colour and texture. Also, a review is under way concerning existing techniques
based on psychological aspects that should reliably represent the way of
thinking of most designers.
Figure 4: Relevance feedback function
TRENDS Prototype 2 software integrates relevance feedback:
the user selected some positive (green frame on the left) and some negative
(red frame on the left) examples; the results are presented on the right.
Semantic text and image search technology
Work on semantic search technology by combined use of text
and visual descriptors has started with the investigation of existing state of
the art algorithms, such as data and feature fusion techniques and bindings of
results on each modality. For the necessities of TRENDS system, the most
adapted method to perform hybrid text and image queries is based in late fusion
of results obtained separately for each modality (see figure 2). This avoids
the very high dimensional vector spaces associated to textual features and the
curse of dimensionality. Moreover, this method is very well adapted when
combining results from different technologies which are not open source (as is
the case in the TRENDS project).
Figure 5: Fusion of results obtained for each modality
The TRENDS system offers text and visual rankings based on
different query formulations: keywords search and query by example. Each
modality provides a complete ordering of the dataset and returns a list of most
fitted elements to the query subject. The work continues with experimenting
several late merging and re-ranking algorithms.
Database and Meta Search Engine
The second version of the database gathered approximately
500.000 good quality of images that illustrate products in many sectors:
automotive, architecture, aeronautics, fashion, sailing, sport, etc.
Figure 6: TRENDS database V2, sector of Architecture (NOVEMBER 2006, 14 Websites, 2,2 Go,
65 000 files)
During the second year of the project, this database has
been widely improved by grabbing fifteen sectors. The evolution of the
statistics for the database is presented in the following table. The third list
refers to the database used for prototype 2. The grabbing for the last version
of the database has begun and will be completed by the end of the year. We can
expect a really rich database of about a million images (on the basis of more
than 6200 websites in 24 sectors).
Table 1: Database results
The table shows that the filtering process removed 87% of
the text and only 26% of the images resulting in a more efficient index. In the
last months major improvements have been realized to automate the grabbing and
filtering processes. New tools have been implemented thus improving the
robustness, the error recovery, and providing statistics. Moreover, the last
version of the grabber is able to process flash websites which was missing in
the previous versions. The core of the meta-search engine enables to test
queries in a Pertimmizer mode. There is not yet a user interface that could allow
wide user testing. The meta-search engine is able to interpret Pertimmizer
queries and emulate web search engines to retrieve results (Linguistic segmentation of a complex
request). The collected results are then processed thanks to the Pertimmizers
(Semantic filtering of the results)
and the resulting scores are used for presenting the better results. This meta-search
engine fully rely on the semantic dimension of Pertimm's products. This
semantic approach is one of the major innovations of TRENDS project. Moreover,
the indexation process has been changed so that the anchors, legends, images
file names are indexed with a higher weight, thus improving the results to user
queries. The system will enable the users to annotate the images. These user
comments will be included in the index so that they will be searched. The user
comments will be shared and editable by all users.
Cutting edge interface
During the second year of the project, the interface of
prototype 2 has been developed. This interface was co-built with the designers,
since the design and development process was as tight as possible to their
expressed requirements gathered through the different creativity sessions and
preliminary tests that took place during the first year. This interface
constitutes the interactive software that allows the user to request and receive
information from the text and image search engines. In comparison with the
interfaces usually used, it can be appreciated that it is graphically advanced,
and it manages a big amount of data every time, however, it is capable of
maintaining a very good response time, totally satisfactory for the user, as it
has been confirmed in the tests.
The user interface of prototype 2 has been developed in C#.
C# is an object-oriented programming language developed and standardised by
Microsoft as part of its platform .NET. It syntaxes results from C/C++ and uses
the model of objects of platform .NET. The integration of the different servers
under the user interface being executed in the client’s side has been carried
out and validated in prototype 2. This prototype 2 proves the feasibility of
the integration of the different modules which constitutes one of the main
achievements of this prototype, and key issue in the final TRENDS software.
Moreover, the user interface of prototype 3, that gathers
all the prototype 2 limitations detected during the end users’ tests and
includes the proper improvements, is even more cutting edge. Its wide
functionality, careful aesthetic and fast interaction with the users turn it
into a useful and nice tool that manages to automate successfully manual actions
in first design stages.
Figure 7: TRENDS User Interface
Architecture and components integration
One of the main innovations in TRENDS lies on the fact that
several servers and modules have to communicate. An intensive work of design
has been performed in order to define the exchange protocols among all these
modules. Prototype 2 proved the feasibility of this architecture. The
performances of the system are good, even with disseminated search servers. A
first and important step has been achieved with the online search on Prototype 2. A permanent work is
conducted to improve the protocols, reduce the weight of the exchanges, and make
them more efficient and less bandwidth consuming. All the communications
between TRENDS servers and client-side IHM will go through the Request
Management Server via HTTP request. The following schema illustrates the
integration of all the modules.
Figure 8: TRENDS Architecture
User Involvement, Promotion and Awareness
Continuous test sessions had been carried out with the end-users
CENTRO RICERCHE FIAT and STILE BERTONE. These included: the initial needs
analysis, the validation of TRENDS functions, the early evaluation of the Graphical
User Interface (GUI) concept and of the non interactive prototype, and finally
the evaluation of the second prototype which is interactive.
The currently main sector targeted by TRENDS is industrial
design. The project website gives an overview of the project outside the
consortium. The TRENDS partners can find more detailed information at by
logging into the secured area of the site. A project flyer is available on the
website in the project website section; this also gives an overview of the
project. In addition, a video demonstration is available on the public part of
the website, showing the main functionalities of prototype 2.
The following papers were published and presented in the
framework of international conferences.
Future Work or Exploitation Prospects, as appropriate
Concerning TRENDS prototype 2, the user centred approach
focused on assessing the interface design with items like efficiency,
satisfaction, aesthetic, affective and cognitive responses to GUI design,
search performance and database quality.
The next year will focus on the development of the final prototype
and of the final software. This will include the current functionalities on the
basis of a new improved and cutting edge GUI also integrating the new functionality
of supported visual and semantic categorisation and semi-automatic pallets
generation. The performance of the full prototype will be evaluated by the
end-users in June-July 2008 and its acceptance will also be validated.
The end-users here will be manufacturers we are currently
working with (CRF and SB), and others such as French and Italian car suppliers. Designers from other industrial design sectors (computers,
PDA’s, mobile phones, fashion), and more widely stakeholders working on markets
linked to the need for cross-lingual and mixed semantic text and image queries
will also be approached.
