Free Essay

Learning English

In:

Submitted By dungpham35
Words 810
Pages 4
Project group “Learning Agents in Dynamic Environments III”

Homework “Reinforcement Learning”
Multiagent Reinforcement Learning

Michael Baumann, Timo Klerx 06-Nov-2012

Foreword
23-Nov-2012, 11:59 PM Hand in your solutions as PDF and the source code (whole eclipse project) as zip archive via eMail. 27-Nov-2012 Presentation and discussion of your results.

Scenario
Consider the scenario shown in Figure Figure 1: The two agents should learn to reach their respective goal area as fast as possible. Although there are explicit start areas, the learned policy should cover all possible states. Admissible actions for the agents are moving one cell in one of the four cardinal directions. If an agent tries to leave the world or crashes into a wall, it remains on its position. The same holds for collisions between agents: on each cell at most one agent is allowed and thus, both agents remain on their positions, if they would collide. This rule implies, that only one agent can pass the small opening at a time! An episode ends either if both agents are situated on their goals region or after imax iterations.

Exercises
1. Discuss the following strategies for giving reward and choose one for your model / your implementation: a) Each agent gets an individual reward, i.e. agent Aj gets reward rj when it reaches goal Aj . b) Both agents get an identical reward r if both agents are on their correct goals.

1

Goal A2

Goal A1

1

2

Start A1

Start A2

Figure 1: Mulitagent scenario. What are pro and cons of these strategies? How can they influence the speed and the quality of learning? Justify your choice. Provide values for the reward function: What reward is given for reaching the goal(s), what value is given for other situations, are there intermediate rewards for partially reached goals? [4 p] 2. What is a useful model for this scenario / the learning task? If it is different to the MDP from SARL, describe these differences briefly and justify your choice. [3 p] 3. What is a useful initialization for learning / evaluation? Discuss the following strategies and choose one for your implementation: a) Agents start only at “their” start areas. b) Agents are placed randomly on arbitrary positions. c) Agents are positioned randomly on arbitrary positions with the following constraint: Agent 1’s starting position has to be on the left side of the obstacle, Agent 2’s starting position has to be on the right side. Justify your choice. [2 p] 4. Which information of the environment have to be passed to the agent? The complete state (“all seeing eye”, each agent knows everything about all agent), a limited view (How small can the sight-range become so that the task remains solvable?) or some kind of abstract state (Is the task still solvable?)? [2 p] 5. How can random exploration be achieved in MARL? Discuss the following strategies and choose one for your implementation: a) Should each agent explore individually? What problem(s) can arise? b) Should the exploration be synchronized, i.e. both agents explore at the same time? How can this be achieved? Justify your choice. [4 p] 6. Implement one (to be precise: the method with your login) of the following approaches to solve the given scenario: [12 p]

2

a) Joint Action Learners [CB98] jflick b) WoLF-PHC [BV02] fmeith c) Distributed Q-Learning [LR00] fabiansc d) FMQ [KK02] mmeier86 e) Team-Q [Lit01] cborn 7. ( ) Compare your results with an approach where each agent executes single-agent Q-Learning and ignores the other agent (Independent Learners). 8. How can MARL tasks be evaluated? What is a useful metric? Justify and implement your choice and plot the results. Use appropriate settings so that your evaluation is reasonable and statistically convincing. [8 p] Remarks: • Although we gave you a fixed number of alternatives in Exs. 1, 3, 5 you may choose a different alternative but please give some reasons and argument for your choice. • Be aware that this scenario may require more episodes of learning.

References
[BV02] Michael Bowling and Manuela Veloso. Multiagent learning using a variable learning rate. Artificial Intelligence, 136:215–250, 2002. [CB98] Caroline Claus and Craig Boutilier. The dynamics of reinforcement learning in cooperative multiagent systems. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98), pages 746–752, 1998. [KK02] Spiros Kapetanakis and Daniel Kudenko. Reinforcement learning of coordination in cooperative multi-agent systems. In Proceedings of the Eighteenth National Conference on Artificial Intelligence (AAAI-02), pages 326–331, 2002. [Lit01] Michael L. Littman. Value-function reinforcement learning in markov games. Cognitive Systems Research, 2(1):55–66, 2001. [LR00] M. Lauer and M. Riedmiller. An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In Proc. of the 17th Intl. Conf. on Machine Learning (ICML 2000), pages 535–542, 2000.

3

Similar Documents

Free Essay

English Learning

...Huang Xu ESL 33B 2/3/2015 My Experience of English Learning As leaning English for almost fifteen years, I have to say I had a lot of experience of learning this language. Yes, I studied English for fifteens, it’s been a long time, but I still can’t speak English fluently. Honestly I don’t think my English is close to “fluently”, I think my English still can describe as “broken” or “fractured”, just like Amy Tan’s mother. The good news is I can feel my improvement, and maybe in years, my English would no longer be “broken”. Any way, as learning English for fifteen years, I always have some interesting things to share. There’s too much to talk about, like my favorite teacher, or the worst teacher I’ve ever met, or the experiences I went through when I first come to America. But because Amy Tan and Elizabeth Wong’s stories, I’d like to share the story about my first English teacher Mrs. Wu and my mother. So let me talk about Mrs. Wu first. I met her in the first year I went to elementary school. She is not the English teacher in school, and actually she was not the real first English teacher I met, but she made me interested in English. We already started to study English in grade one. However my parents thought the English that I learn in elementary school was too basic, and they thought English will be very useful. So they find me an English class outside the school, I met Mrs. Wu there. I didn’t want to have the class at first. I have no wish to learn a foreign language...

Words: 1274 - Pages: 6

Premium Essay

Observation In Learning English

...This paper concentrates on the observations made during lessons. It also focuses on how learning English for the first time as a language can be taught in a way that would be better and easy to understand. The students that I will deal with are those of grade one and two. Learning activities highly influence on how a student appreciate. Making sure that learning activities are of high quality which can be coupled with situations gives the students’ different perspectives which they can appreciate. They are several relevant skills that are a student is supposed to practice during learning. The skills are clarification, value analysis and problem-solving. The experience which students get during learning are the ones that give the opportunity...

Words: 1881 - Pages: 8

Premium Essay

Online Learning Vs Learning English

...Learning English has become one of the necessity to face rapid changes in this globalization era. The desire to learn English is increasing and it resulted in increasing the number of courses for English, either online course or face to face course. Both types of learning require instruction from teacher, have the same purpose to give knowledge and make the students can speak English fluently and write properly in English. However, learning English online and learning English face to face, they differ from one to another in term of flexibility, convenience, time efficiency, and also cost. Traditional class is not flexible as it requires the students and teacher to attend a particular place at specific time. For instance, if students of traditional...

Words: 860 - Pages: 4

Free Essay

The Importance of Learning English  

...The importance of learning English   English is the most widely used  language in the world. Therefore, having great English skills will have numerous advantages. Luckily, I have realized the importance of learning English and worked hard on it since I was a child. However, 13 years of learning English experience didn’t lead me to success, but being a person who can hardly communicate in English and definitely  unable to write an essay. Once I came to the USA, I knew that I had made too many mistakes in learning English. Those mistakes are translating English into different languages, don’t learn vocabulary as much as I can, and don’t read and write enough.    First of all, every language is unique. They usually have different grammar and use of vocabulary, which means learning English by translating it into different languages will lead to an inevitable defeat. For example, I have worked hard on learning a lot of vocabulary by translating them into Chinese. I thought it was a good idea to build up my vocabulary in order to read and write better. However, those translation processes were hugely disappointing. At that time, I failed to express my idea accurately by those translated vocabulary. As a consequence, good grade and improvement seemed untouchable. Also, when I either wrote or read, I had to use Chinese to think about the passage. As a result, I didn’t do it well and did it very slowly. When I came to the USA, I desperately and completely stopped translation. Just a year of time...

Words: 488 - Pages: 2

Premium Essay

Learning English

...Learning English through children's literature Submitted by admin on 18 December, 2002 - 12:00 This article is about the British Council's Young Learners Centre in Paris and how they use children's literature in their teaching of English   * The role of stories and storytelling in language teaching * Selecting story books * Pupil responses * Personal and professional development of teachers * Other support materials * Books referred to in this article   The role of stories and storytelling in language teaching Once upon a time and not so very long ago in the capital city of France, a teaching centre for little children and not so little children was opened. One little child and then two and then three and then many, many more came along. And so our story unfolds ….. There was a little red hen, a meerkat in trouble, a brown bear, a black elephant and a white elephant, a very hungry caterpillar, Spot the dog, a clever tortoise, a big, roaring, yellow, whiskery lion, a kangaroo from Woolloomooloo and many more. These are just some of the colourful characters from children's literature who have helped children aged 5 - 10 attending holiday classes at the British Council's Young Learners Centre in Paris learn English. These weekly courses take place each afternoon for two hours. The educational value of using stories and the technique of storytelling has always been undisputed throughout the world. Now more and more English as a foreign language (EFL) teachers of...

Words: 9557 - Pages: 39

Premium Essay

My Strength and Weakness in Learning English

...My Strengths and Weaknesses in Learning English In all levels of education, the quality of every learner’s grasp and understanding always varies since each student obtains knowledge from their educators in different ways. Additionally, it also reflects that students will have dissimilar strengths and weaknesses which can actually be seen through their study while they are all together learning in the same class. From what I have remarkably noticed since the start of learning English, I constantly find myself having many strengths and weaknesses listed below. To begin with, grammar is my goodness in learning English. I can analyze and catch up the grammatical structures, tenses, and usages sharply and clearly whenever teachers or lecturers have explained them to me. Also, I rarely make any mistakes in grammar exercises since I am quite careful and thoughtful when building sentences or choosing the right answers. Moreover, good time-management is my another best virtue in studying English. I can separate between my study time and pleasure time distinguishably. Furthermore, I usually sort out a study-calendar, which enables me to submit every homework and assignment to teachers or lecturers on time. To sum up, grammar and good time-planning are my strengths in learning English. Besides, listening is my feebleness in studying English. I am not able to understand the whole conversation well once the native speakers speak very fast. Also, I never get good scores in listening test...

Words: 355 - Pages: 2

Premium Essay

Immigrants Learning English

...Immigrants Learning English What does immigrant mean? Immigrant is a person who comes into a country to live there. When immigrants come into whatever country, they have to learn a new culture and new language to integrate society in that country. In fact, language is the key of the culture, so learning new language in new country will help immigrants understand more about the typical culture’s in new country; each country has a different languages, and language is the most important thing to communication with native speakers so that it helps people understand each other. For example, if immigrants come to France, they have to learn French; immigrants move to China, they can learn Chinese; or if immigrants are resident in Spain, they must learn Spanish etc. Likewise, immigrants, who come into the U.S, can learn English; learning English is very helpful to immigrants in the U.S because they can communicate with somebody by English, get more opportunities and support their children. First, learning English helps immigrants can communicate with native speakers or...

Words: 1690 - Pages: 7

Premium Essay

Learning English

...00 U.S. | | Introduction In Going Graphic: Comics at Work in the Multilingual Classroom, Cary has written a useful resource for language teachers eager to supplement typical textbooks and graded or simplified readers with less traditional authentic materials. Going Graphic will provide teachers with a theoretical framework, handy responses to skeptics, a wealth of useful classroom activities, and plenty of recommended comics. Teachers new to comics will find enough here to make good use of them with second language learners. If you are already using comics, you will probably find some new ideas. Style Cary is serious about advocating the use of comics for second language learning, but this book is not overly academic in tone. In keeping with both the medium and the author’s personal connection to the...

Words: 1644 - Pages: 7

Premium Essay

Essay About Learning English Language

...Focuses on the relevance of learning the English language to the Saudi society , Upon means of communication and nations' conflict of interests, learning has become an urgent need in which judged by contemporary cultural differences and the openness. As in this research, we will introduce the necessity of learning English. Introduction As we are in the era of globalization and the entire world has become a little village upon the exchange of interests and that people of all nations communicate in daily basis, learning English has become a necessity in the whole world. Nowadays, most countries, including Saudi Arabia, save no effort in the matter of teaching English. It is the language of today in which it's needed in all aspects of life....

Words: 788 - Pages: 4

Free Essay

A Study on the Performance and Study Habits Employed in Learning English

...perception on their study habits and to evaluate whether study habits of Grade 5 and Grade 6 students in SMA has positive relationship with their academic performance in Learning English. Measures of study behaviors, also called study skills, strategies, or techniques, can serve as diagnostic tools to help instructors identify students in need of additional help and can also provide a better awareness of a student’s strengths and weaknesses and, correspondingly, ways to optimize his or her learning. Study behaviors can be broadly defined as behaviors functioning to acquire, organize, synthesize, evaluate, remember, and use information (Crede & Kuncel, 2008; Gettinger & Seibert, 2002). Such behaviors include time management; goal setting; selecting what, how, and where to study; taking good notes; reading; and self-testing. Researchers have divided the many specific study behaviors into four main categories: repetition-based (e.g., flashcards), cognitive-based (e.g., studying with a friend), procedural (e.g., time management), and metacognitive (e.g., taking quizzes to test self-knowledge). Today, a lot of students are failing their subjects or have low grades on English. These may be occurring due of lack of study skills/habits. Good study skills minimize failure and enable students to take advantages of learning opportunities. The researchers believe to achieve good study skills; one must have a desire to learn and to...

Words: 1454 - Pages: 6

Free Essay

English Learning

...* Formål * Formålet med dette forsøg er, at bestemme identiteten af en ukendt alkohol, ud fra nogle parametre, og ved brug af udelukkelsesmetoden. * Teori * Til rådighed er der fem nummereret flasker, med en alkohol i hver. I nedenstående skema er alkoholerne, samt informationer om disse, skrevet ind. Alkohol | Type | Strukturformel | Densitet | Kogepunkt | Ethanol | Primær alkohol | CH3 – CH2 – OH | 0,789 g/mL | 78,29° C | Propan-2-ol | Sekundær alkohol | CH3 – CH – CH3 OH | 0,780925 g/mL | 82,30° C | Butan-1-ol | Primær alkohol | OH − CH2 – CH2 – CH2 – CH3 | 0,8095 g/mL | 117,73° C | Butan-2-ol | Sekundær alkohol | CH3 – CH – CH2 – CH3 OH | 0,8063 g/mL | 99,51° C | 2-methylpropan-1-ol | Primær alkohol | OH – CH2 – CH – CH3 CH3 | 0,8018 g/mL | 107,89° C | * Den flaske man vælger, indeholder en ukendt alkohol som har visse egenskaber. De egenskaber der skal bestemme identiteten af alkoholen er: a) Densitet (g/mL). b) Kogepunkt (° C). c) Alkoholens evne til at blive oxideret af Beckmanns blanding: Beckmanns blandingen består af kaliumdichromat i svovlsur opløsning og er god til oxidationer af primære og sekundære alkoholer. Hvis der havde været en tertiær alkohol blandt de udvalgte alkoholer, ville denne ikke kunne oxideres af Beckmanns blanding og der ville da, ikke kunne ses nogen farveændring. Dette skyldes at tertiære alkoholer ikke kan oxideres, fordi det carbon-atom der er bundet...

Words: 1910 - Pages: 8

Premium Essay

Important of Learning English

...helps us communicate and show our feelings, ideas, moods, attitudes and expressions at our will. Everybody knows that English is the most popular language nowadays, and it has been for centuries. Besides the popularity, everybody should know that it is one of most important and crucial skills to have for personal success, fun and any other part of life. Which leads to another point, if we could make all people realize that the English language is that important, they would simply try harder to learn it in schools or in their own time. Above all it is a global formula for learning. With this in mind, if one realizes the importance and benefits of something he will try harder to learn it. Besides this realization of the benefits, if we like the subject we are studying, we will most likely learn it faster and with positivity. The English language is an international means for communication and expressing one's feelings anytime and anywhere. Benefits of English language As the quote from Benjamin Lee Whorf says, it shapes the way we think and determines what we can think about, we can conclude that language is in fact a thought and a thinking process. Therefore the conclusion of the quote is that without language, we can not think. Even if we think it is difficult to share and express our thoughts, it is the consequence of thinking. After all, the English language is our best friend" when considering connecting to the outer world. In view of this, it is the main language of...

Words: 349 - Pages: 2

Premium Essay

Learning English in Young Age

...Learning English in Young Age: Children learn English as another Language Introduction There seems to be no optimum age for learning a language, according to linguistics experts. Singleton (1995) writes, “There is no point at which vocabulary acquisition can be predicted to cease.” Adults have several advantages over children when it comes to learning foreign language, especially English. Research by Harold B. Dunkel and Roger A. Pillet in 1957 assessed both adult beginning learners of French and elementary students learning French. In their studies Dunkel and Pillet found that the adult students performed better in written and oral tests, but worse on pronunciation. Adults tend to maintain both native and foreign languages at a high level while children who learn a second language can lose proficiency in the first language. But it is universally acknowledged that learning English, since early childhood would contribute positively to the child’s language acquisition. Researchers at University College London have discovered that people who are bilingual have more gray matter (neurons or brain cells) in the language region of their brains. The earlier a child learns English, the more advanced his or her brain becomes in language acquisition. This reinforces the idea that it is better to learn English in the early years. That is why I believe that learning English in young age is better than older one. The importance of language for young children The early years are recognized...

Words: 1402 - Pages: 6

Free Essay

English Grammar Learning Jouranal

...ENGL 220 ------------------------------------------------- Learning Journal Week 1 Intro grammar: Units & classes Outline of Content Intro grammar: Units & classes grammatical units: clauses, groups/phrases, words, morphemes, different classes of units: nominal, verbal, adverbial, conjunctive, prepositional Map of grammar: composition Rank of units: [grammar] clause — group / phrase — word — morpheme; [graphology] sentence — sub-sentence — word — letter; Rankshift (downranking) Class: clause: major / minor; group: nominal / verbal / adverbial; word: — nominal: determiner / adjective /noun — verbal: verb (auxiliary / lexical verb) — adverbial: adverb Professor: Christian M.I.M. Matthiessen Core Readings: Thompson (2004), Chapter 2: “Recognizing clauses and clause constituents” DFG, Chapter 1, pp.1-5 Part I : Summary of what I’ve leant (i) Difference between Traditional Grammar & Systemic Functional Grammar Grammar is a subsystem of language for constructing meanings in the form of wordings. Traditional grammarians gradually developed accounts of word classes (part of speech) since words belonging to different classes have different sets of forms and of the grammatical categories associated with the forms of different word classes—e.g. number and case for nouns, and person and tense for verbs. It is the study of traditional grammar. However, if we want to understand how grammar is organized and operates as a...

Words: 1586 - Pages: 7

Premium Essay

A Study on the Importance of Grammar in Learning English

...GRAMMAR IN LEARNING ENGLISH In India English continues to be the medium of instructions in colleges and universities. It is also the language of the administration. English should be most useful “library language” in higher education and our most significant window on the world. English plays an important role in our national life. It was of great importance in the British India. It plays an important role in various fields. So grammar plays a vital role in the field of learning language. So working knowledge of grammar is required for a sound basis of language- ungrammatical use will create disorder, misunderstanding of ideas, and bad impression among the listener or the reader. In the words of Dr. West, “grammar is not a code of rules like etiquette and table manners, a statement of convention, it summarizes what is done by cultured people, and like etiquette, and it is a state of a constant”. NEED AND SIGNIFICANCE OF THE STUDY The most important ingredient of learning English is grammar. Because, when we come to learn a new language, we need to study its grammar. Grammar is the study of words and the ways words work together. Any person who communicates using a particular language is consciously or unconsciously aware of the grammar of that language. Students usually find difficulties in learning grammar and most of them hate grammar sessions. They do not realize that it is the back bone of language. Pupils at secondary schools also have difficulties with learning complex...

Words: 1012 - Pages: 5