If nothing is stated about the presentation language then the presentation is in Swedish.
2023-06-08 - SaS |
Kamerabaserad trafikstyrning - en realtidssimulering av trafikdetektering
Rikard Arnemo, Oscar Castenäs
Grundnivå (16hp)
kl 10:15, IDA John von Neumann (På svenska)
[Abstract]The traffic control of today is primarily controlled using buried induction loops that detect vehicles passing over them. These detectors break relatively often and require excavation of the roadway to repair. These reparations take time, generate obstructions in the flow of traffic and lead to further societal costs. This paper explores the possibility of using cameras directly mounted to the traffic lights of an intersection as a substitute for induction loops. The camera-based solution is required to perform on a similar level to the solution based on induction loop for this to be plausible. This paper constructs a model of an intersection and uses that model to evaluate if this is possible or not. The intersection is constructed in Unreal Engine 5 and is based on a real-world intersection from Linköping, Sweden. The model includes a representation of the induction loops present in the real-world intersection. It also includes an implementation of a camera-based vehicle detection solution where the cameras are mounted in a realistic way throughout the simulated intersection. The measurement Fördröjningstid is used to measure the performance of the different solutions. Fördröjningstid is the added time a vehicle that passes through the intersection experiences because of the traffic lights. The paper then discusses the findings and draw a conclusion based on the results of the tests done. |
2023-06-08 - SaS |
Uppskattningar av utsläpp från anläggningsmaskiner - En förstudie
Fredrik Halldin
Grundnivå (16hp)
kl 11:15, IDA Alan Turing (På svenska)
[Abstract]The construction industry is one of the main causes of global warming, which is responsible for 20% of the total annual emissions in Sweden. In order to reduce the emissions from construction equipment, more energy efficient equipment can be used on construction sites, which Skanska Rental is striving towards. The purpose of this work was to find out how emissions from construction equipment can be estimated and presented to customers who rent the equipment. In this prestudy, one group interview with customers was conducted, as well as a literature review of five different calculation methods for emissions, and two individual interviews regarding data collection. The group interview showed that customers would like to know how much emissions the equipment produces, and that they can improve their sustainable decision making with that information. The literature review showed that all five calculation methods can be used to estimate the emissions, but that they have different pros and cons. The European environment agency’s tier 3 calculation is seen as the best suited for Skanska rental. The main reason for this is the high level of detail, which allows specific equipment to be compared to each other. The method mainly uses information which can be acquired from manufacturers, which make the calculations scalable. The data which might need to be collected is the operating time of the equipment, and what work is being done during that time. This can be used to produce a more representative load factor, which can improve the estimation. How to gather this data was discussed in the individual interviews, where installing sensors is seen as a viable option. Finally, more studies are encouraged to be carried out with the life cycle perspective included, and that manufacturers should produce life cycle analyses of their equipment. This is because electric equipment has been identified as a future development within the field, which requires the life cycle perspective when estimating emissions. |
2023-06-08 - SaS |
Exploring the use of simulation and visualisation in a large-scale game
Jessie Chow, Jenny Rudemo
Avancerad (30hp)
kl 13:15, IDA Alan Turing (På svenska)
[Abstract]"As the climate crisis becomes increasingly apparent, efforts are done to encourage and nuance discussion on energy system transition. One way this is done is through games that combine role-play, board game elements and computer simulations, i.e. megagames. The aim of this thesis is to address the topic of how visualisations and simulations can be designed and used to nuance such discussions within a megagame.
Starting with a broad perspective on sustainable development through the United Nations' Sustainable Development Goals, bioenergy was identified as a topic of interest through surveys and observations at two megagame sessions.
Based on literature on forest dynamics and historical data from Swedish forests, a simple simulation model of a biomass system was designed. The model was designed to simulate extraction of biomass and absorption of carbon dioxide for a forest with four age categories. Data on carbon dioxide emissions, absorption, biomass extraction and forest age distribution was generated from the biomass model and an already existing energy system model through simulations. The data was visualised in several types of plots and representations according to design criteria derived from observations, surveys and similar work.
The resulting visualisations provide different levels of detail and can be used both for general announcements to all participants of the megagame and to support specific player groups as well as the game leaders." |
2023-06-08 - SaS |
Telemetry for Debugging Software Issues in Privacy-sensitive Systems
Kasper Landgren, Payam Tavakoli
Avancerad (30hp)
kl 15:15, Alan Turing (In English)
[Abstract]Traditionally, when debugging a software system, developers rely on having access to the input data that caused the error. However, with data privacy concerns on the rise, it is becoming increasingly challenging to rely on input data as it might be sensitive or even classified. Telemetry is a method of collecting information about a system that can offer assistance. This thesis proposes a telemetry solution for debugging systems where input data is kept private. The telemetry solution was implemented in a geographical 3D visualization system and evaluated based on two aspects: its effectiveness in assisting the developers working on the system to locate if and where a problem has occurred and its ability to deduce the input data that generated the output. The results indicate that the telemetry solution is successful in helping developers identify system errors. Additionally, the system's privacy is maintained, as it is not feasible to directly ascertain the input data responsible for the output. Telemetry can be a useful tool for developers, making the debugging process more effective while protecting sensitive input data. However, this might not be the case for different customers or systems. |
2023-06-09 - SaS |
Rust async for thread-per-core packet processing
Josef Olsson, Isak Stenström
Avancerad (30hp)
kl 08:15, IDA John von Neumann (In English)
[Abstract]When building a packet processing system, it is common to use frameworks that are written in C and built upon DPDK Eventdev, which is what Ericsson AB currently uses. However, using Rust async for this task could improve the developer experience and efficiency. This thesis explores the use of Rust async for creating reliable packet processing systems by implementing Rust Packet Processing Performance Prototype (RPPPP), which uses the Rust async runtime Glommio. RPPPP was implemented with support for using both centralized and decentralized scheduling. Two applications, that closely mirror the functionality of two existing DPDK Eventdev programs, were developed using RPPPP. One of the existing programs is a public DPDK Eventdev example application that uses a centralized scheduler, and the other is an internal Ericsson benchmarking application that uses a decentralized scheduler. The performance of DPDK and RPPPP was compared by evaluating the performance of all four applications for different core counts. The results show that the Rust application with the decentralized scheduler has a throughput that is between 20% and 50% of its DPDK counterpart, while its latency is far worse. The Rust application with a centralized scheduler performs significantly worse than any of the other applications when using more than 4 cores, both in terms of throughput and latency, but this is deemed to not be the true potential of the performance of this approach. Though not fully conclusive, the result of this thesis indicates that using Rust async for packet processing systems is currently not a viable approach. |
2023-06-09 - AIICS |
Developing High-level Behaviours for the Boston Dynamics Spot Using Automated Planning
Nisa Andersson
Avancerad (30hp)
kl 09:15, Kurt Gödel (In English)
[Abstract]Over the years, the Artificial Intelligence and Integrated Computer Systems (AIICS) Division at Linköping University has developed a high-level architecture for collaborative robotic systems that includes a delegation system capable of defining complex missions to be executed by a team of agents. This architecture has been used as a part of a research arena for developing and improving public safety and security using ground, aerial, surface and underwater robotic systems.
Recently, the division decided to purchase a Boston Dynamics Spot robot to further progress into the public safety and security research area. The robot has a robotic arm and navigation functionalities such as map building, motion planning, and obstacle avoidance. This thesis investigates how the Boston Dynamics Spot robot can be integrated into the high-level architecture for collaborative robotic systems from the AIICS division. Additionally, how the robot's functionalities can be extended so that it is capable of determining which actions it should take to achieve its desired goals considering its capabilities and current state. The study was divided and done in an iterative approach.
The system was tested in various scenarios and demonstrated the ability to plan and accomplish the intended goal. However, there were occasions when achieving the goal proved challenging due to various limiting factors. |
2023-06-09 - SaS |
Optimizing E‐commerce Logistics: A Multi‐Metric Approach to the Bin Packing Problem
Anton Magnusson, Vilhelm Melkstam
Avancerad (30hp)
kl 10:15, IDA Donald Knuth (In English)
[Abstract]The optimization of packaging selection in logistics, particularly within the realm of ecommerce, offers numerous potential advantages, such as a reduction in environmental impact and decreased costs. This thesis addresses the problem of allocating items to the minimum number of packages, known as the bin packing problem, by proposing various heuristics. We develop and assess heuristics for assigning products to groups, while heuristics for accommodating these groups within packages are derived from previous research. These heuristics are evaluated within a commercial context, taking into account factors such as delivery cost, environmental impact, and their applicability in real-time systems. Our findings indicate that optimal solutions for smaller orders can be ascertained within a reasonable
timeframe, while even rudimentary heuristics yield satisfactory results. It was determined that a key attribute of an effective solution was lowering the number of packages used, as this correlates with reduced shipping costs and environmental impact. |
2023-06-09 - ADIT |
Preserving privacy in cloud services by using an Explainable Deep-Learning Model for Anomaly Detection
Shiwei Dong
Avancerad (30hp)
kl 13:15, John von Neumann (In English)
[Abstract]As cloud services become increasingly popular, ensuring their privacy and security has become
a significant concern for users. Cloud computing involves Data Service Outsourcing and Computation
Outsourcing, which require additional security considerations compared to traditional computing.
Detecting hazardous user data in the cloud is challenging, as cloud service providers must
also protect user data privacy. Many researchers have proposed deep learning models for anomaly
detection in cloud services, with LSTM networks being particularly effective at detecting timely
anomalies through sample reconstruction.
This project aims to detect anomalies in time series cloud data to help maintain the availability
of cloud services and produce timely alerts. Due to the specification of cloud data, this model
only analyzes basic timely traffic data as input without digging into the sensitive content generated
by users. To protect privacy, we experimentally deploy a homomorphic encryption mechanism that
enables classification without reading plaintext. The homomorphic encryption mechanism is a novel
privacy-preserving scheme utilized in machine learning which can preserve information distribution
while fully encrypted. If widely deployed, this scheme benefits the tackling of Machine Learning
as A Service privacy issues. Furthermore, by using feature importance as weights for multivariate
integration, we explain the decision-making process of our LSTM-based model, thus enhancing its
transparency and trustworthiness.
To evaluate the feasibility of our model, we conduct several experiments on two datasets using
accuracy as the primary evaluation metric. The first is a plain time series with individual anomalies,
where our model successfully detects all anomalies without accessing privacy features and outputs
reconstruction errors as anomaly scores. The second is a real-world network traffic dataset with
DoS attacks, where our model can effectively detect attack periods. Our approach shows promise
in improving cloud security and can be helpful for cybersecurity departments, network security
companies, and cloud service providers. Lastly, this project is individual work which suggests that
there can be further enhancements. Researchers interested in similar topics can further apply and
improve the model. Also, this model is not independent, which can be integrated into security
systems more extensively. |
2023-06-09 - SaS |
Choosing authentication protocol for digital signatures
Pontus Kågström
Grundnivå (16hp)
kl 15:15, IDA Donald Knuth (SaS) (In English)
[Abstract]More and more companies are working toward digitizing their workflow and this has increased the necessity of digital signatures. An important part of digital signatures is
the authentication process which is heavily regulated for Swedish government agencies by DIGG, DIGG only allows the use of Security Assertion Mark-up Language(SAML) for authentication but are looking into also allowing OpenID Connect(OIDC) and together withSwedish OIDC working group producing a specification. This thesis is looking into this
preliminary specification and exploring if OIDC can do everything that SAML can do inregards of digital signatures, and if the inclusion of OIDC would render SAML obsolete.
This is explored by implementing OIDC in twoday’s services that follow DIGG’s specifications to see if there are needs that OpenID Connect cannot meet. From the restriction in thethesis there was nothing that SAML can do that OIDC can’t do, On the contrary their are features in OIDC that SAML could not match. The inclussion of OIDC would probably not make SAML obsolete unless customers use-cases evolve to include the features that SAML could not match." |
2023-06-09 - AIICS |
Autonom 3D-utforskning med dynamiska hinder
Ludvig Widén, Emil Wiman
Avancerad (30hp)
kl 15:30, Alan Turing (På svenska)
[Abstract]The advancements within robotics in recent years has increased the demand for sophisticated algorithms that can tackle the challenges associated with building robust and safe autonomous systems. The objective of 3D-exploration is to enable a robot to explore an unknown environment with a high degree of accuracy while minimizing time and path length. In static environments, planners such as RH-NBVP and AEP have been widely studied. However, in dynamic environments where obstacles like pedestrians or vehicles can appear, existing solutions either use dynamic motion planners for obstacle avoidance or treat dynamic objects naively like DEP, which only considers obstacles within its field of view. This thesis examines how dynamic obstacles can be included in the planning process while performing 3D-exploration to make more advantageous decisions, in terms of both efficiency and overall safety. Considering the uncertain dynamics in the environment is crucial for a robot to act safely, and hence be deployable in a real-world scenario. The suggested solution has been evaluated with other 3D-exploration planners as benchmarks. Experiments demonstrate that static planners struggle in dynamic environments. However, with a simple reactive motion planner, most collisions can be avoided. This thesis presents an extension to the static planner AEP that takes dynamic obstacles into consideration. The proposed solution samples more safe routes to a goal and takes into account historical observations of dynamic obstacles. Finally, a novel potential volume gain is implemented to predict explored volume in the future. |
2023-06-12 - HCS |
Evaluation of alternative UX design principles for To Do lists
Nazir Elpustaty, Malin Widén
Avancerad (30hp)
kl 08:30, Alan Turing (In English)
[Abstract]As government authorities are getting more digitalized, the variety of case management systems is increasing. Therefore, the competition is getting tougher for companies that develop such systems and the goal to create a system that both fulfills its purpose and is easy to use is getting more important. Human-centered design is used to evaluate how general design principles work on To Do lists in a case management system. A pre-test survey is sent out to current users of a case management system to understand their requirements, what works for them and what does not. Then, three prototypes were developed from the answers to the pre-test survey. The prototypes were then evaluated by the system’s users during online evaluation sessions. The result was that the functionalities requested by the users and that were developed with the help of design principles were more popular than functionalities inspired by only the principles. Also, functionalities that were less of a change were more desired to be implemented in the system directly. To conclude, the designer is not the user and a pre-test is essential to understand the users’ perception of the system. The pre-test should catch what amount of redesign is reasonable and what should not be changed. Further, design principles should be followed along the way, not to inspire changes that were not requested by the users, but rather to develop requested changes. |
2023-06-12 - HCS |
Diggery
Rayan Bayat, Mohamad Shafeah
Grundnivå (16hp)
kl 10:15, John von Neumann (På svenska)
|
2023-06-12 - ADIT |
Using Vocabulary Mappings for Federated RDF Query Processing
Juliette Winneroth
Avancerad (30hp)
kl 13:00, Babbage (In English)
[Abstract]Federated RDF querying systems provide an interface to multiple autonomous RDF
data sources, allowing a user to execute a SPARQL query on multiple data sources at once
and get one unified result. When these autonomous data sources use different vocabu-
laries, the SPARQL query must be rewritten to the vocabulary of the data source in order
to get the desired results. This thesis describes how vocabulary mappings can be used
to rewrite SPARQL queries for federated RDF query processing. In this thesis, different
types of vocabulary mappings are explored to find a suitable vocabulary mapping repre-
sentation to use in formulating an approach for query rewriting. The approach describes
how the SPARQL subqueries and solution mappings can be rewritten in order to handle
heterogeneous vocabularies. The thesis then presents how the query federation engine
HeFQUIN is extended to rewrite the federated queries and their results. A final evaluation
of the implementation shows how implementing a query rewriting approach can improve
the federated query engine’s execution times. |
2023-06-12 - AIICS |
Compact Representations of State Sets in State Space Search
Hugo Axandersson
Avancerad (30hp)
kl 13:15, Alan Turing (In English)
[Abstract]Modern day technological advancements are moving at a rapid pace. In the field of Artificial Intelligence, algorithms are becoming ever faster and process larger amounts of data. These fast algorithms call for data structures that can store this processed data compactly. This premise also holds true in the AI subfield of planning. In the common planning approach of state space search, found states are memorized as to not unnecessarily revisit them. Research has put a big focus on improving the speed of state space searches which in turn leads to a lot of states being stored. A crucial bottleneck then occurs when memory runs out due to storing these large amounts of states. This is where this project, with its exploration of compact state set representations, comes into the picture.
This project's focus is on exploring memory usage for planning by state space search. More specifically, the project investigates memory compact state set representations for an A* state space search's closed- and open lists. It was hypothesized that the closed list would be the larger of the two which is why a focus was put on testing compact representations of that state set. Results from this project confirm this hypothesis as it is shown that the closed list is the largest and most critical of the two (although the differences between the two become increasingly small for strong heuristics).
Four different state set representations were tested for use as closed lists in an A* algorithm. The state set representations were: Level Ordered Edge Sequences (LOES), compressed LOES (cLOES), Binary Decision Diagram (BDD) and explicit representation. A primary focus was put on exploring the LOES data structure because of the limited amount of research done on the data structure. Explicit representation was used as the main point of comparison with it being a very commonly used standard in state space search.
The results showed that LOES and cLOES proved strong replacements to explicit representation. Especially in use cases where memory compactness is more critical than speed such as in embedded systems. Additionally, even though strong heuristics lessened the effect of efficient state set representations, there's still notable advantages to be found for big tasks where the closed list grows large enough. |
2023-06-13 - SaS |
Exploring Decision Diagrams for Task Scheduling: Empirical Evaluation of Construction Methods for Relaxed Diagrams
Dennis Berntsson
Avancerad (30hp)
kl 10:15, IDA Alan Turing (In English)
[Abstract]Decision diagrams have recently emerged as a promising approach for difficult scheduling problems, along with other challenging discrete optimization problems. Decision diagrams can offer a compact representation of the solution space, and has the ability to capture complex constraints that are hard to model or express in other techniques. This thesis explores two standard construction methods for relaxed decision diagrams, top-down construction and incremental refinement. The techniques are compared on their ability to handle large-scale scheduling problems with multiple time windows and precedence constraints. The construction methods are evaluated on several metrics, including generated bound, execution time, and the size of the diagram, on instances of the problem with up to 200 tasks. The results show that incremental refinement generates smaller diagrams with good bounds, at the expense of execution time. |
2023-06-13 - SaS |
Ökad upptäckbarhet av APIer i ett storskaligt monorepo med hjälp av fjärrindexering av källkod
Niklas Granander
Avancerad (30hp)
kl 10:15, IDA Muhammad al-Khwarizmi (ADIT) (På svenska)
[Abstract]Some of the largest corporations in the world use a single, multi-project spanning repository, or monorepo, to handle large amounts of source code. When developing software in a monorepo, engineers rarely need nor can have all components of the repository in their local development environments without sacrificing IDE performance. Since indexing the full source code would be infeasible on a single machine, engineers at Spotify can use IDE project slicing when working in a monorepo, which means they only access and index a part of the repository at once. This thesis investigates what challenges in terms of discoverability exist when working in a monorepo with IDE project slicing, and how remote source code indexing can be used to mitigate some of these challenges. This research was initiated by interviewing engineers to find out what challenges existed and followed by the design, development and evaluation of a prototype of a tool to mitigate these challenges. The prototype used source code processed by Kythe as data and exposed an interface to search and explore the indexed source code. The prototype was evaluated by letting engineers try to find and learn about APIs with and without the prototype. The evaluation showed that the prototype was an effective aid for finding APIs and that there is potential in using remote source code indexing to help developers learn how to use unfamiliar APIs by providing example usages. The results also include a set of recommendations for future research or for owners of large-scale monorepos to consider when addressing discoverability. |
2023-06-13 - SaS |
System Testing of AI – An Overview on Vulnerabilities in Image Classification Systems and System Verification.
Ahmad Hanash
Avancerad (30hp)
kl 10:15, IDA Kurt Gödel (AIICS) (In English)
[Abstract]As ML (Machine Learning) and deep neural networks get more used in many systems, the need to understand and test such systems becomes more actual. Moreover, when de- signing a new system that contains ML models, what should be considered to ensure the safety of a system becomes inevitable especially when the system needs to consider such nonfunctional requirements. Therefore, the need to address the potential problems and weaknesses in such a system and the discussion of potential strategies to face these problems and weaknesses was presented in this thesis, taking into consideration the context of such defiance strategies and the ML model purpose in a complex software system. Lastly, the result presented in this thesis shows that considering using a ML system in a complex software system with high safety requirements requires adopting software methodologies such as MLOps (Machine learning operations) to keep such systems under monitoring and give suggestion to how to test and verify an ML model integrated into larger software system. |
2023-06-13 - SaS |
Test Case Selection in Continuous Integration Using Reinforcement Learning with Linear function Approximator
Younus Salman
Avancerad (30hp)
kl 13:15, IDA John von Neumann (In English)
[Abstract]Continuous Integration (CI) has become an essential practice in software development, allowing teams to integrate code changes frequently and detect issues early. However, the selection of appropriate test cases for CI remains a challenge, as it requires balancing the need for thorough testing with the limited time and resources available. This study proposes a practical and lightweight approach that leverages reinforcement learning with a linear function approximator for test case selection in CI. Several models are created where each one focuses on a different feature set. The method aims to optimize the selection of test cases by learning from past CI outcomes, both the historical data and the coverage data, and dynamically adapting the models, by using a linear function approximator. Through experimentation and comparison between the models and with existing methods, the study demonstrate which feature set is optimal and efficient. The result indicate that reinforcement learning with a linear function approximator using coverage information can effectively assist in selecting test cases in CI, leading to enhanced software quality and development efficiency. |
2023-06-13 - HCS |
Developing a jump platform exergame where high scores maintain a high degree of movement
Linus Olsson, Alfred Spjut Molin
Grundnivå (16hp)
kl 13:15, Alan Turing (På svenska)
|
2023-06-14 - SaS |
The Impact of Gamification on User Engagement in a Second- hand E-commerce platform
Yen Dinh, Maya Fogelberg
Avancerad (30hp)
kl 10:15, IDA John von Neumann (In English)
[Abstract]Using game elements in a non-game context, called gamification has grown popular recently and can be seen in education, health, and on e-commerce websites. In this thesis, the use of game elements, specifically badges is evaluated on a secondhand website. The gamification component is implemented in three steps, a paper prototype, a digital prototype, and an implementation in ReactJS. All iterations are iterated using CRM with feedback from the client and with Lean UX. The game elements are evaluated using A/B testing and interviews with users from the A- and B tests.
The results show that gamification can have a positive impact on user engagement on a secondhand website. It also shows that it can be incorporated to communicate the core values of the business. The A/B test indicated that there is room for improvement in the gamification component, specifically by more clearly showing that badges are clickable and by adding an explanation of how they can be achieved. These improvements would likely contribute to a better user experience and increase user engagement with the gamification component. |
2023-06-14 - AIICS |
Modeling of Human Preferences without Humans – A Study on Data Augmentation for Large Language Reward Models
Oskar Hallström
Avancerad (30hp)
kl 15:15, John von Neumann (In English)
[Abstract]Recent large language models such as chatGPT, GPT-4, Claude, and Bard have gained enormous interest thanks to their impressive performance and helpfulness for humans. These large language models are pre-trained on huge amounts of text to predict the next word, and then further refined through reinforcement learning from human feedback. In this pro- cess, a reward model is first trained to capture human preferences and then used to pro- vide a reward signal in the reinforcement learning loop. In this manner, the large language model is steered towards the behavior desired by humans. These reward models, not only used within reinforcement learning, are trained on ranked text data, which in its simplest form consists of samples with a prompt and a pair of two ranked continuations. As this data exists in limited amounts and is expensive to annotate for humans, automatic generation of ranked training data becomes an interesting area of research. By investigating how existing and new generation techniques for ranked data impact the accuracy of reward models with respect to a dataset based on human preferences, this thesis contributes with new insight on the impact of these generation techniques as well as highlighting their potential. In particular, this thesis introduces AI distractors and AI continuation pairs. When combined with previous techniques, AI distractors give an improvement over these techniques, resulting in that replacing a partition of human-ranked data with generated data can give slightly better performance than with the human-ranked data as is. With AI continuation pairs, this thesis takes a first rudimentary step into observing the capabilities of reward models trained on no data ranked by humans to model human preferences. |
2023-06-15 - SaS |
Design and evaluation of a plain MPI-based cluster execution backend for the SkePU-3 skeleton programming framework
Alexander Zeijlon
Avancerad (30hp)
kl 08:15, IDA John von Neumann (In English)
[Abstract]SkePU 3 is a framework for parallel program execution that uses higher order functions called skeletons, which provide a layer of abstraction between user code and the parallel implementation it provides through its backends. The backend that enables SkePU to run on an HPC cluster has a slowdown of a factor two. This
reduces the viability of SkePU as an alternative for HPC, and as such, warrants an investigation.
Programs written in SkePU are sequential-looking, single-source C++ programs where skeleton calls can transparently execute on multiple different types of processing units, such as CPU cores, GPUs and clusters, using different backends.
In this thesis, a strategy for improving the performance of SkePU on clusters is presented, and with it, the design and implementation of a new cluster backend that is simpler and more closely integrated with the non-cluster SkePU code base. Runtime measurements show that the new cluster backend sees a relative speedup of about a factor of two, which effectively eliminates the slowdown. |
2023-06-15 - SaS |
Automatisk bedömning av programmeringsuppgifter med språkmodeller
William Johansson
Grundnivå (16hp)
kl 09:00, Alan Turing (På svenska)
[Abstract]Under 2023 har chattbottar drivna med språkmodeller tagit världen med storm. Språkmodeller tränade på en blandning av programmeringskod och text har visat sig kapabla att skriva program samt att tolka kod. I detta arbete undersöks om modellen gpt-3.5-turbo från OpenAI kan bedöma kod-inlämningar från studenter och sätta liknande poäng som en lärare. Två olika scenarier testas, i det ena fallet ska modellen först beskriva och analysera implementeringen av uppgiften innan poäng ges, i andra fallet sätts poäng direkt. För att få modellen att svara på ett givet format och bedöma rätt detaljer i koden används en instruktion som förbättras iterativt. Utmaningar uppstår i att begränsa längden på svaret från modellen samt få modellen att konsekvent följa formatet. Instruktionen anpassas med uppmaningar som upprepas i olika uttryck för att komma tillrätta med utmaningarna. Totalt bedömer modellen 50 inlämningar från en tentamen som skickas in av ett automatiskt testprogram. Resultaten visar att totalpoängen som GPT-3.5 har satt i genomsnitt är cirka 20% (av maxpoäng) ifrån den mänskliga bedömningen i båda scenarierna. Modellen gav generellt högre poäng än lärarbedömningen och satte endast i ett fall underkänt på en godkänd inlämning. |
2023-06-15 - SaS |
En jämförande studie mellan MySQL och MongoDB mot HCL Notes-databasen
William Hallengren
Grundnivå (16hp)
kl 09:30, IDA Alan Turing (På svenska)
[Abstract]Databasbelastningstestning är en avgörande aspekt vid utförande av olika tester på en databas. Själva belastningstestningen ger en bra bild av hur en databas presterar under belastning, både när det gäller motståndskraft och prestanda. I denna studie sätter vi upp 3 olika databaser vilket är: MySQL, MongoDB och HCL Notes-databasen. För att se till att testningsverktyget för databasen bättre utför ändringar i databasen så sattes en REST API upp. Denna REST API implementerar alla nödvändiga funktioner för att skapa, läsa, uppdatera och ändra data. I experimentet används notifikationer som data. Belastningstestningsverktyget användes för att skapa planer som utför alla nödvändiga funktioner såsom skapa, läsa, uppdatera och ändra för en notifikation, för att sedan utföra denna plan flera gånger vilket resulterar i en stor mängd förfrågningar till databasen. Svarstiden för alla förfrågningar dokumenterades och jämfördes. Detta gav en klar vinnare, MySQL som hade lägst responstid för alla test med olika mängder förfrågningar och visade aldrig tecken på att krascha. HCL Notes-databasen var långsammast och klarade nästan av att köra alla tester medans MongoDB presterade bättre än HCL Notes-databasen när det kommer till svarstid men klarade inte av att köra alls lika många tester som de andra databaserna. Sammanfattningsvis, MySQL presterade bäst och i liknande scenarion som detta, är den bästa databasen. |
2023-06-15 - SaS |
Just Answer, en Quiz-applikation: Utvärdering av automatisk anslutning med Wi-Fi och Bluetooth för förbättrad användarinteraktion
Anton Knudsen, Oskar Lund
Grundnivå (16hp)
kl 10:00, IDA Alan Turing (På svenska)
[Abstract]Syftet med detta examensarbete är att undersöka möjligheten att utveckla en konceptuell Quiz-applikation vid namn Just Answer, samt utvärdera dess begränsningar. Målet är att förenkla
interaktionen för användare genom att eliminera traditionella anslutningsmetoder som till exempel QR-koder eller sifferkoder, där istället tekniker som Wi-Fi och Bluetooth används för att låta automatisk anslutning ske. För att skapa en grund för arbetet genomfördes en litteraturstudie, och baserat på den utvecklades två prototyp-applikationer som användes för att genomföra tester. Resultaten av dessa visar på att konceptet är genomförbart, där användning av Wi-Fi och Bluetooth möjliggör till enklare och snabbare interaktion än traditionella anslutningsmetoder. |
2023-06-15 - AIICS |
Grafikkortsaccelererade DBMS för finansiell analys
Emil Strömberg
Avancerad (30hp)
kl 10:15, John von Neumann (På svenska)
[Abstract]I en värld med stora mängder tillgänglig data, och speciellt inom ett användningsområde som finansiell analys, där kursdata uppdateras i realtid, är det av vikt att verktyg erbjuder en så god användarvänlighet som möjligt. Detta innefattar både ett brett erbjudande av nyckeltal, samt maximerad relevans av nyckeltalen, däribland genom att vara nyligen uppdaterade.
Då moderna grafikkortssystem erbjuder god prestanda, samt att problemet (finansiell nyckeltalsberäkning) lämpar sig väl för parallellisering, tack vare att indatamängden i hög grad är oberoende, har grafikkortsaccelererade databassystem identifierats som ett intressant område för utveckling av användarvänliga verktyg vid finansiell analys.
Utifrån en teoristudie har egenskaper hos grafikkort och de därefter följande designval som uppstår för grafikkortsaccelererade databassystem kartlagts, och respektive designvals för- och nackdelar har utvärderats. Därefter har de grafikkortsaccelererade databassystemen HeavyDB och Kinetica identifierats som lämpliga verktyg utifrån popularitet och lämplighet för problemet, samt testats över en uppsättning nyckeltal och datamängder för att jämföra prestandaegenskaper och kostnadsbild för respektive verktyg.
Implementationerna visade överlag goda prestandaegenskaper för användningsområdet, där Kinetica för de nyckeltal som testades visade en viss prestandafördel jämfört med HeavyDB. Dessutom visade Kinetica övrig nämnvärd funktionalitet som både underlättar vid integrering av Kinetica i existerande system och skapar flexibilitet vid utformning av användarlösningar. Samtidigt medför dock Kinetica en licenskostnad och är påtagligt dyrare än HeavyDB som är av öppen källkod. |
2023-06-15 - SaS |
Utvidgning av SQL-bibliotek i kompilatorn Storm - En jämförande studie av SQL-satser och datatyper i olika databashanterare
Hanna Häger, Simon Westerberg Jernström
Grundnivå (16hp)
kl 10:40, IDA Alan Turing (På svenska)
[Abstract]Detta examensarbete undersöker möjligheterna att vidareutveckla ett SQL-bibliotek i den interaktiva kompilatorn Storm. Vidareutvecklingen innebär tillägg av stöd för fler databashanterare, då SQL-biblioteket för närvarande endast stödjer SQLite. För att möjliggöra tillägget krävs en generell SQL-syntax som är kompatibel med olika databashanterare. Utmaningen som uppstår vid framtagandet av en generell syntax är databasernas variationer och avvikelser från varandra och SQL-standarden. Denna studie fokuserar på en jämförelse mellan databashanterarna med syfte att identifiera gemensamma SQL-satser och datatyper. Målet är att ta fram ett gemensamt subset som kan stå till grund för den generella syntaxen i SQL-biblioteket i Storm. Utifrån studiens resultat kan det fastställas att framtagning av en generell SQL-syntax i Storm är genomförbar. Utöver den jämförande studien innebär dessutom arbetet en implementering av stöd för databasen MariaDB i Storm. |
2023-06-15 - STIMA |
A machine learning approach to classify colorectal cancer genes and immune genes of humans using Next-Generation RNA Sequence (RNA-Seq)
Lawrence Thanakumar Rajappa
Avancerad (30hp)
kl 11:00, Charles Babbage (In English)
[Abstract]In this study, we aim to predict invasiveness of cancer subtypes by using a supervised machine learning approach. We will base on the next-generation RNA sequence data of cancer cells and cancer-associated immune cells to classify the cancerous cells from immune cells in a given cancer sample taken from cancer patients and compare the classification/prediction performance of machine learning models such as SVM, decision tree, random forest, hierarchical clustering, etc. and choose the model that performs better. The evolution of genomic sequencing technology has led researchers to perform various experiments to find a cure for various genetic diseases such as cancer. The advent of Artificial Intelligence, especially in the healthcare sector has paved the way to numerous solutions such as drug discovery, cost-effective treatment for patients, etc. which would be beneficial for pharmaceutical organizations, hospitals, and researchers to perform their experiments at a fast pace. Further, Next-generation RNA sequencing data has helped to find the relationship between gene expression profiles and different developmental stages of a cell or a disease in an efficient and cost-effective approach with higher accuracy. There are various methodologies mentioned in various literature related to a cure for cancer, still, the fatality rate of cancer cannot be reduced. The machine learning approach helps us to understand the complexities and processes of cancer samples that lead to the development of effective cancer treatment/drugs in a reliable, fast, and efficient way. |
2023-06-15 - SaS |
Statistical Analysis of Large Datasets: Subset Selection and Performance Evaluation using Elasticsearch
Angella Ritta Arslanian, Ritta Sulaiman Youssef
Grundnivå (16hp)
kl 11:20, IDA Alan Turing (In English)
[Abstract]This thesis explores the use of Elasticsearch in determining the minimum size of a subset that yields statistically significant results compared to larger datasets, as well as the impact of varying subset sizes on the time performance of the subset selection process using Elasticsearch and Kibana. The research methodology involves generating test data, randomly sampling subsets of different sizes, performing statistical analysis with Elasticsearch, conducting T-tests, analyzing time performance, and utilizing Metricbeat for CPU performance analysis. The study found that a subset of 500 documents consistently demonstrates statistical significance compared to a larger dataset of 100,000 documents, with low p-values indicating its effectiveness and representativeness of the larger dataset. The investigation on varying subset sizes' impact on time performance using Elasticsearch and Kibana showed minimal variation in average processing time (22-24 milliseconds), highlighting the efficiency of the subset selection process, unaffected by the number of documents. The analysis of CPU usage patterns unveiled a positive correlation between CPU usage and time performance. Larger subset sizes were found to enhance computational efficiency by effectively utilizing CPU resources, whereas smaller subset sizes exhibited higher CPU usage. Among the subset sizes examined, the subset size of 300 demonstrated the highest CPU usage. |
2023-06-15 - HCS |
Investigations of free text indexing using NLP
Alfred Sundstedt
Avancerad (30hp)
kl 13:00, Donald Knuth (In English)
[Abstract]As Natural Language Processing progresses societally and applications like OpenAI obtain larger popularity in society, businesses encourage integration of NLP into their systems. Both to improve the user experience and provide users with their requested information. For case management systems, a complicated task is to provide the user with relevant documents, since customers often has large databases containing similar information. This presumes that the user needs to match the requested topic perfectly. Imagine if there was a solution to search for context, instead of formulating the perfect prompt, via established NLP models like BERT. Imagine if the system understood its content. This thesis aims to investigate how a free text index can be improved using NLP from a user perspective and implement it. Using AI to help a free text index, in this case, Apache Solr, can make it easier for users to find the specific content the users are looking for. It is of interest to see how the search can be improved with the help of NLP models and present a more relevant result for the user. NLP can be used to improve user prompts, known as queries, and assist in indexing the information. The task is to carry out a practical investigation by configuring the free text database Apache Solr, with and without NLP support. This is investigated by learning the search models its content, let the search models provide their relevant search results, for some user queries, and evaluating the results. A hybrid search model of OpenNLP and BERT turned out to be the best solution. |
2023-06-15 - STIMA |
Machine Learning Algorithms for Personal Loan Default Prediction: An Empirical Study
Martin Gustafsson
Avancerad (30hp)
kl 14:00, John von Neumann (In English)
[Abstract]Credit risk evaluation is a central problem for any financial institution. Default prediction is a significant portion of credit risk evaluation and is therefore supremely important for the success of a financial institution’s loan portfolio. That loan portfolio is often heavily imbalanced, in favour of non-defaulting loans. This thesis uses the imbalanced loan portfolio of a Swedish consumer credit lender to provide empirical evidence for the performance of four machine learning algorithms. The imbalanced nature of the data is used to provide further empirical evidence for the performance implications of SMOTE-NC. By comparing models with varying amounts of SMOTE-NC to models without it, we investigate whether the synthetic minority samples generated by SMOTE-NC yield performance improvements.
The four machine learning algorithms selected for this thesis are logistic regression, neural networks, random forest, and XGBoost. Interestingly, the empirical evidence presented suggests that no algorithm is able to outperform the simple logistic regression algorithm. Furthermore, SMOTE-NC appears to reduce the performance of algorithms while causing models to become overconfident in predicting default. |
2023-06-16 - AIICS |
Improving News Accessibility Through Chatbots – An Exploration of the Effectiveness of Using Chatbots to Increase Access to Up-To-Date News
Abedalhkeem Najeeb, Johan Nordling
Avancerad (30hp)
kl 08:15, Alan Turing (In English)
[Abstract]People today are used to and expect a dynamic and personalized information flow;
this also holds true for news content. This study investigated the potential of using LLMs,
document retrieval techniques, and text style transfer models to develop an up-to-date,
truthful and trustworthy Swedish news chatbot. Various lexical and semantic document retrieval models were assessed, with the hybrid approach combining BM25 as the first stage
retrieval model and the fine-tuned Sentence-BERT model as the reranker model the most
effective. The chatbot’s truthfulness varied depending on the article retrieval model used.
A prompt-engineered GPT-4 model outperformed a fine-tuned GPT-3 model in transforming text to a journalistic text style. Insights from this study can inform the development of
LLM-powered news chatbots in low-resource languages like Swedish, enhancing accessibility and engagement in journalistic content. |
2023-06-16 - ADIT |
Increasing Efficiency and Scalability in AWS IAM by Leveraging an Entity-centric Attribute- & Role-based Access Control (EARBAC) Model
Pontus Jönrup, Rasmus Karlsson
Avancerad (30hp)
kl 12:30, Charles Babbage (In English)
[Abstract]Cloud computing is becoming increasingly popular among all types of companies due to its inherent benefits. However, because of its infrastructure, it might be difficult to manage access rights between users and resources. To address these difficulties, Amazon Web Services (AWS) provides Identity and Access Management (IAM) and features that support the use of different access control models, for example Role-based Access Control (RBAC) and Attribute-based Access Control (ABAC). Access control models are used for authorisation within systems to decide who gets access to what. Therefore, to determine what constitutes an efficient and secure access control model, a thorough study of background material and related work was conducted. It was found that flexibility and scalability were also two important factors when designing access control models. Furthermore, by conducting a survey and designing an access control model for AWS through various iterations, a new access control model called EARBAC was developed. In an experiment comparing it with another access control model, the EARBAC model was found to be both efficient and secure, in addition to its flexibility and scalability. This suggests that the model is useful when developing cloud infrastructures in AWS. |
2023-06-21 - SaS |
How to design a digital guide app in Flutter/Firebase with a focus on ease of use.
Malcolm Sävmarker
Grundnivå (16hp)
kl 10:15, Allan Newell (På svenska)
|
2023-06-21 - ADIT |
Verification Method for Time of Capture of a Rolling Shutter Image
Alexander Johansson, Filip Johansson
Avancerad (30hp)
kl 13:00, Charles Babbage (In English)
[Abstract]Modern automotive systems increasingly depend on camera sensors to gather safetycritical data used in driver-assisting features of the system. These features can consist of for example lane-keeping assist and automatic braking where the sensors register objects within certain distances. When these camera sensors gather information, the time of the image is critical for the calculation of speeds, distances, and size of any potential registered object in the frame. Limitations of bandwidth and computing in such vehicles creates a need to use special cameras that do not capture the whole image simultaneously but instead capture the images piecewise. These cameras are called rolling shutter cameras. This puts pressure on defining when an image was captured when different parts of the image were captured at different points in time. For this thesis, this point in time is defined as the chronological middle point of the camera starting to capture an image and when it has collected the final part of it.
This thesis performs a mapping-study to evaluate methods to verify the timestamp of an image generated from rolling shutter cameras. Further this thesis proposes a new method using multiple digital clocks and presents its performance using a proof-of-concept implementation to prove the method’s ability to accurately represent time with submillisecond accuracy. |
2023-06-21 - HCS |
A Small Step for a Sensor
Johan Fallström
Avancerad (30hp)
kl 13:15, IDA Alan Turing (På svenska)
[Abstract]In this paper, a technical overview will be provided for a developed mobile exergame, with a particular focus on its movement tracking. By utilizing spatial movement readout from the AR algorithm, we’ve managed to create an easy to use exergame that allows the user to track their horizontal movement on a very limited area. In contrast to a more conventional approach, our solution can work indoors, and can be applied to vertical motion tracking as well. The method used while developing this exergame was meant to ensure that it was well suited for its target group, but it didn’t include a thorough examination of other alternatives to our AR usage. This means that our solution should be reserarched further to better understand its relevance in the field, while we’ve shown with a practical example how it can be utilized. |
2023-06-21 - SaS |
Comparison of infrastructure as code (IaC) frameworks for defining cloud infrastructure in code from a developer’s perspective
Daniel Karlsson
Avancerad (30hp)
kl 13:15, IDA Allen Newell (HCS) (In English)
[Abstract]The cloud has become much more important and relevant to the IT industry in recent years. Instead of buying and maintaining their own physical servers, companies now often opt for renting servers and services from cloud providers. These servers can be thought of as abstract units of computing power that can dynamically allocated or disposed of depending on need. Configuring such an infrastructure can be very complex and challenging. This is where Infrastructure as Code (IaC) comes into play. Instead of having to manually create and configure each resource and their web of connections, Infrastructure as Code (IaC) can be used to describe the desired state of the infrastructure in a declarative way as code. The IaC tools will then take care of creating and configuring the resources in the cloud to match that desired state. There exist quite a few different IaC tools. This thesis will go through the IaC tools AWS CDK and Pulumi and compare them to provide you with insight to help you choose which tool will fit you best. The focus will lie on the readability aspect of the tools. This comparison will be done by implementing a specific infrastructure using each of the tools and then comparing the implementations using a survey where people vote on which one they prefer as well as by using the metrics Lines of Code (LOC), Cyclomatic Complexity (CC) and Cognitive Complexity (COG). The command line interface (CLI) of the tools were also briefly evaluated to see if they followed some of the recommended guidelines. |
2023-06-21 - AIICS |
Energy Efficient UAV Coverage Path Planning In The Forestry Industry
Gustav Stappe Renner
Avancerad (30hp)
kl 15:00, Alan Turing (In English)
[Abstract]Using UAVs to cover areas of interest is common within Coverage Path Planning. Several different approaches and solutions exist to solving this general problem. However, not much work has been done on modeling the energy consumption of a UAV and using it to find energy-efficient paths. The contributions from this thesis work are twofold; firstly, a comparison of the state-of-the-art Coverage Path Planning algorithms BA* and inward spiral on even and uneven areas and an evaluation of them based on total flight time and battery consumption, and secondly, presenting an energy-efficient Coverage Path Planning solution that uses a generalized energy model as a heuristic cost in the A* search algorithm. This solution was coined A* with energy constraints. Through real-world flights, results showed that the differences between the state-of-the-art algorithms in different areas were minor. This was most likely due to the minimal differences in altitude between the areas. A* with energy constraints was evaluated through simulations alongside the other algorithms on a very hilly area. Results showed that the proposed solution was estimated to be more energy efficient than BA* but less than inward spiral. |
2023-06-22 - HCS |
Implementation och utvärdering av en webb-baserad lärplattform
Mathilde Hennings, Emil Nilsson
Avancerad (30hp)
kl 10:15, Alan Turing (På svenska)
[Abstract]Modern education is more and more dependent on technical solutions to assist student learning. Systems specifically designed for this purpose are called learning management systems and provide the link between students and teachers, in the sense that a significant part of their communication is handled within the system. With a learning management system as a core part of a course, the demands on the usefulness and usability of the system are high. The usefulness of an in-house developed system was evaluated in a real world scenario from a teacher's perspective. The evaluation concludes that the implemented changes increased the usefulness of the system and simplified the work for the teachers. |
2023-06-22 - ADIT |
Shooting for Perfection – Leveraging Basketball Movement Data to Predict Shot Accuracy
Niklas Samuelsson
Avancerad (30hp)
kl 13:15, Babbage (In English)
[Abstract]This paper examines what factors determine if a basketball shot will be accurate or not. To do this, movement data containing the positions of all players at all times during the game is utilized. Play-by-play data is also used to extract information about each shot. Through the combination of these datasets, the conditions on the court at the exact moment of the shot are found. From this, different features representing the situation are generated for each shot. These features are then used to train models to predict the outcome of shots. The features are also ranked based on their importance for the prediction. The study concluded in models able to predict the outcome of shots 61% of the times, which is similar to what previous works have been able to achieve. The biggest takeaway is the importance of selecting the correct shot moment to be able to predict the outcome of a shot, a topic which has not received much attention in previous works. A new, model-based approach for doing this is proposed for further research. |
2023-06-22 - ADIT |
Comparison of lossy and lossless compression algorithms for time series data in the Internet of Vehicles
Joseph Hughes
Avancerad (30hp)
kl 13:15, John von Neumann (In English)
[Abstract]As automotive development advances, connectivity features are continually added to vehicles that in conjunction form an Internet of Vehicles. It is in the interest of vehicle manufacturers to collect data from their fleets, however, the volume of the generated data is too large to feasibly be transmitted to a server due to the performance required and the cost of network usage.
The purpose of this paper is to investigate how these issues can be alleviated by compressing the data before transmission. We first perform a literature study to identify state of the art compression algorithms for time series compression algorithms that run online and provide max-error guarantees. We then choose a subset of lossless and lossy algorithms that are implemented and benchmarked with regards to their compression ratio, resource usage and reconstruction error when used on time series that exhibit a variety of data features. Finally, we ask whether we are able to run a lossy and lossless algorithm in succession in order to further increase the compression ratio.
The literature study identifies a diverse range of compression algorithms. Out of these, the algorithms Poor Man's Compression - MidRange (PMC-MR) and Swing filter are selected as lossy algorithms, and Run-length Binary Encoding (RLBE) and Gorilla are selected as lossless algorithms. The results are mixed for the lossy algorithms, which both result in high compression ratios but excel in different areas. In contrast, Gorilla outperforms RLBE in nearly all aspects. |
2023-06-22 - SaS |
Compiler Testing by Random Code Generation
Victor Löfgren
Avancerad (30hp)
kl 13:15, IDA Donald Knuth (SaS) (In English)
[Abstract]Most software projects today are written using programming languages. Compilers in turn translate programs written in these higher level languages into machine code, executable on actual hardware. Ensuring that these compilers function correctly is therefore paramount. Manually written test suites make sure that compilers functions correctly for some inputs, but can never hope to cover every possible use case. Thus, it is of interest to find out how other testing techniques can be applied.
This project aims to implement a random test program generator for Configura Magic (CM), a proprietary programming language used at Configura. Our tool is inspired by the widely successful C program generator Csmith. It is implemented by randomly generating an abstract syntax tree (AST) and unparsing it to produce correct code.
Our tool found about 9 bugs in the CM compiler, Configura Virtual Machine (CVM), during its development. CVM was instrumented to get code coverage data after compiling programs. Compiling the CVM test suite (CTS) and Configura's main product CET (Configura Extension Technology) cover about 23% and 19% of the compiler respectively, while compiling programs generated by our tool only covers about 6%. But on the other hand, it uniquely covers about 0.2% that is not covered by the other programs.
A backend for our tool that generates C-code was also implemented, to compare it against Csmith. The results show that on average (100 program generations 30 times, for a total of 3000 programs), our tool gets about 45% coverage while Csmith gets about 50% on the small C compiler TinyCC. Although our tool was mildly successful in finding bugs, the comparison between it and Csmith shows its potential to be even more effective. |
2023-08-28 - AIICS |
Learning Partial Policies for Intractable Domains on Tractable Subsets
Viktor Carlsson
Avancerad (30hp)
kl 15:15, Alan Turing (In English)
[Abstract]The field of classical planning deals with designing algorithms for generating plans or sequences of actions that achieve specific goals. It involves representing a problem domain as a set of states, actions, and goals, and then developing search algorithms that can explore the space of possible plans to find the one that satisfies the specified goal. Classical planning domains are often NP-hard, meaning that their worst-case time complexity grows exponentially with the size of the problem. This means that as the number of states, actions, and goals in the problem domain increases, the search space grows exponentially, making it very difficult to find a plan that satisfies the specified goal.
This thesis is concerned with investigating these NP-hard domains, specifically by simplifying these domains into ones that have a polynomial solving time, creating a general policy for the simplified domain, and then attempting to apply this policy onto the original domain. This would create a partial policy for the original domain, and the performance of this policy can be measured in order to judge its effectiveness. |