{"refrec":{"BRefID":391535,"RR":"<b>Wang, W.; Cao, X.J.; Gonzalez-Garcia, A.; Yin, L.H.; Hagemann, N.; Qiao, Y.Y.; Ratti, C.; Rus, D.</b> (2023). Deep reinforcement learning based on tracking control of an autonomous surface vessel in natural waters, <b><i>in</i></b>: <i>Conference proceedings ICRA 2023, 29<sup>th</sup> May – 2<sup>nd</sup> June, 2023: International Conference on Robotics and Automation.</i> pp. 3109-3115. <a href=\"https://dx.doi.org/10.1109/ICRA48891.2023.10160858\" target=\"_blank\">https://dx.doi.org/10.1109/ICRA48891.2023.10160858</a>","BEntID":389282,"PublicFlag":1,"CheckedFlag":1,"wosflag":0,"vabbflag":0,"RefStringPartII":", <b><i>in</i></b>: <i>Conference proceedings ICRA 2023, 29<sup>th</sup> May – 2<sup>nd</sup> June, 2023: International Conference on Robotics and Automation.</i> pp. 3109-3115. <a href=\"https://dx.doi.org/10.1109/ICRA48891.2023.10160858\" target=\"_blank\">https://dx.doi.org/10.1109/ICRA48891.2023.10160858</a>","DocTypID":17,"DocType":"Book chapters","MarineFlag":1,"FreshFlag":0,"BrackishFlag":0,"TerrestrialFlag":0,"Authorstring":"Wang, W.; Cao, X.J.; Gonzalez-Garcia, A.; Yin, L.H.; Hagemann, N.; Qiao, Y.Y.; Ratti, C.; Rus, D.","OrigTitleTranslFlag":0,"Authorstringtrunc":"Wang, W. <i>et al.</i>","Englishabstract":"Accurate control of autonomous marine robots still poses challenges due to the complex dynamics of the environment. In this paper, we propose a Deep Reinforcement Learning (DRL) approach to train a controller for autonomous surface vessel (ASV) trajectory tracking and compare its performance with an advanced nonlinear model predictive controller (NMPC) in real environments. Taking into account environmental disturbances (e.g., wind, waves, and currents), noisy measurements, and non-ideal actuators presented in the physical ASV, several effective reward functions for DRL tracking control policies are carefully designed. The control policies were trained in a simulation environment with diverse tracking trajectories and disturbances. The performance of the DRL controller has been verified and compared with the NMPC in both simulations with model-based environmental disturbances and in natural waters. Simulations show that the DRL controller has 53.33% lower tracking error than that of NMPC. Experimental results further show that, compared to NMPC, the DRL controller has 35.51% lower tracking error, indicating that DRL controllers offer better disturbance rejection in river environments than NMPC.","AbstractOtherLang":null,"BibLvlCode":"AM","StandardTitle":"Deep reinforcement learning based on tracking control of an autonomous surface vessel in natural waters","OrigTitleLangCode":"en","OrigTitleLangCodeExtended":"eng","OrigTitleLangID":15,"DateLastModified":{"date":"2024-12-10 01:33:17.368041","timezone_type":1,"timezone":"+01:00"},"UserAccessRight":null,"UserAccID":null,"AuthorKeywords":null,"OtherDescriptors":null,"Notes":null,"AnaPub":2023,"MonPub":null,"DateUpdate":"2024-04-06","DateCreate":"2024-03-26","SecASFANote":null,"ConfID":null,"PeerRev":0,"VlizCoreFlag":1,"WoScode":"WOS:001036713002071","VABBcode":null,"OpenAcc":0,"DOI":"10.1109/ICRA48891.2023.10160858"},"refs":null,"anarec":{"AnaID":391535,"PubliDate":2023,"Pagination":"3109-3115","XtraPublOfAnaID":null,"ISBN":"979-8-3503-2366-5","Volume":null,"Issue":null,"BRefMon":391683,"BRefMonRR":"(2023). Conference proceedings ICRA 2023, 29<sup>th</sup> May – 2<sup>nd</sup> June, 2023: International Conference on Robotics and Automation. IEEE: United Kingdom. ISBN 979-8-3503-2366-5; e-ISBN 979-8-3503-2365-8. [diff. pag.] pp. <a href=\"https://dx.doi.org/10.1109/ICRA48891.2023\" target=\"_blank\">https://dx.doi.org/10.1109/ICRA48891.2023</a>","BRefXtra":null,"BRefXtraRR":null,"SerBRefID":null,"SerRR":null,"StandardTitleSer":null,"ISSN":null,"AbbrevSer":null,"StandardTitleMon":"Conference proceedings ICRA 2023, 29<sup>th</sup> May – 2<sup>nd</sup> June, 2023: International Conference on Robotics and Automation","StartPage":3109,"Pages":7,"ToPubliDate":null,"BRefBibLvlCode":"M","SerNotes":null},"monrec":null,"serrec":null,"relations":null,"relationsRev":null,"addrec":null,"othpubs":null,"ownerships":null,"authors":[{"AutName":"Wang","Firstname":"Wei","Initials":"W.","Affiliation":"MIT, Comp Sci & Artificial Intelligence Lab CSAIL, Cambridge, MA 02139 USA.","Discriminator":null,"CorporateFlag":0,"BEntID":389282,"AutID":563604,"OrderNr":1,"DegrID":null,"EditorFlag":0,"CorrespFlag":0,"IllustratorFlag":0,"ReviserFlag":0,"TranslatorFlag":0,"InsAcronym":null,"InsFSN":null,"ORCID":null,"PersID":null,"InsID":null},{"AutName":"Cao","Firstname":"Xiaojing","Initials":"X.J.","Affiliation":"Beijing Univ Posts & Telecommun BUPT, Sch Artificial Intelligence, Intelligent Percept & Comp Res Ctr, Beijing 100876, Peoples R China.","Discriminator":null,"CorporateFlag":0,"BEntID":389282,"AutID":563606,"OrderNr":2,"DegrID":null,"EditorFlag":0,"CorrespFlag":0,"IllustratorFlag":0,"ReviserFlag":0,"TranslatorFlag":0,"InsAcronym":null,"InsFSN":null,"ORCID":null,"PersID":null,"InsID":null},{"AutName":"Gonzalez-Garcia","Firstname":"Alejandro","Initials":"A.","Affiliation":"MIT, SENSEable City Lab, Cambridge, MA 02139 USA.","Discriminator":null,"CorporateFlag":0,"BEntID":389282,"AutID":563607,"OrderNr":3,"DegrID":null,"EditorFlag":0,"CorrespFlag":0,"IllustratorFlag":0,"ReviserFlag":0,"TranslatorFlag":0,"InsAcronym":null,"InsFSN":null,"ORCID":"0000-0001-5575-0843","PersID":42219,"InsID":null},{"AutName":"Yin","Firstname":"Lianhao","Initials":"L.H.","Affiliation":"MIT, Comp Sci & Artificial Intelligence Lab CSAIL, Cambridge, MA 02139 USA.","Discriminator":null,"CorporateFlag":0,"BEntID":389282,"AutID":563610,"OrderNr":4,"DegrID":null,"EditorFlag":0,"CorrespFlag":0,"IllustratorFlag":0,"ReviserFlag":0,"TranslatorFlag":0,"InsAcronym":null,"InsFSN":null,"ORCID":null,"PersID":null,"InsID":null},{"AutName":"Hagemann","Firstname":"Niklas","Initials":"N.","Affiliation":"MIT, SENSEable City Lab, Cambridge, MA 02139 USA.","Discriminator":null,"CorporateFlag":0,"BEntID":389282,"AutID":563611,"OrderNr":5,"DegrID":null,"EditorFlag":0,"CorrespFlag":0,"IllustratorFlag":0,"ReviserFlag":0,"TranslatorFlag":0,"InsAcronym":null,"InsFSN":null,"ORCID":null,"PersID":null,"InsID":null},{"AutName":"Qiao","Firstname":"Yuanyuan","Initials":"Y.Y.","Affiliation":"MIT, SENSEable City Lab, Cambridge, MA 02139 USA.","Discriminator":null,"CorporateFlag":0,"BEntID":389282,"AutID":563612,"OrderNr":6,"DegrID":null,"EditorFlag":0,"CorrespFlag":0,"IllustratorFlag":0,"ReviserFlag":0,"TranslatorFlag":0,"InsAcronym":null,"InsFSN":null,"ORCID":null,"PersID":null,"InsID":null},{"AutName":"Ratti","Firstname":"Carlo","Initials":"C.","Affiliation":"MIT, SENSEable City Lab, Cambridge, MA 02139 USA.","Discriminator":null,"CorporateFlag":0,"BEntID":389282,"AutID":563614,"OrderNr":7,"DegrID":null,"EditorFlag":0,"CorrespFlag":0,"IllustratorFlag":0,"ReviserFlag":0,"TranslatorFlag":0,"InsAcronym":null,"InsFSN":null,"ORCID":null,"PersID":null,"InsID":null},{"AutName":"Rus","Firstname":"Daniela","Initials":"D.","Affiliation":"MIT, Comp Sci & Artificial Intelligence Lab CSAIL, Cambridge, MA 02139 USA.","Discriminator":null,"CorporateFlag":0,"BEntID":389282,"AutID":563615,"OrderNr":8,"DegrID":null,"EditorFlag":0,"CorrespFlag":0,"IllustratorFlag":0,"ReviserFlag":0,"TranslatorFlag":0,"InsAcronym":null,"InsFSN":null,"ORCID":null,"PersID":null,"InsID":null}],"mapdetails":null,"datasets":null,"monographs":null,"monparts":null,"serparts":null,"BEntOpen":null,"BEntPrivate":null,"availability":[{"BInstID":398970,"LibID":36,"BRefID":391535,"EmbargoDate":null,"FullEmbargoDate":null,"PhysMedID":16,"hasOCRd":null,"ShelfLocCode":"398970","RFID":null,"PaidValue":null,"Medium":"Server","Description":"Interne VLIZ documenten","Acronym":"VLIZ","Library":"Vlaams Instituut voor de Zee","DutchTerm":"Non-open access","URL":null,"ClassifID":228,"Classification":"Non-open access","ReqLink":1,"ClassifTypID":3,"URLLocation":"https://www.vliz.be/imisdocs/publications/","SubDir":1,"InternalReq":1,"LoggedInReq":1,"Disclaimer":"Disclaimer_VLIZ_Intern","DutchDisclaimer":"<p>Deze publicatie is enkel beschikbaar voor persoonlijk gebruik binnen de Innovocean site <br />en mag op geen enkele manier verder worden verspreid.</p>","FileFormat":".pdf","FileDescr":"pdf","InsPub":1,"InsID":36,"FileFormID":6,"LendableFlag":1,"PublicFlag":1,"orderLib":"A","Notes":null,"AccConID":null,"AccessConstraint":null,"LicURL":null}],"litstyles":[{"LitStyID":3,"Style":"Conference paper"}],"thespers":null,"arch2discl":null,"SERpubls":null,"MONpubls":[{"PublName":"IEEE","Place":"United Kingdom"}],"pictures":[],"thestermsPath":null,"thestermsASFA":null,"taxtermsASFA":null,"geotermsASFA":null,"collections":null,"conf":null,"proj":null,"Physdatasets":null,"spcols":{"222":{"SpName":"BMB - Belgische Mariene Bibliografie","SpColID":222,"ParSpColID":null,"TopParID":null,"ShortName":"BMB","URLLocation":null,"LibID":36,"OpenRepoFlag":null,"SpTypID":null,"TopParIDNotWebsite":null,"SpColPath":"BMB"}},"doi":null,"publs":null,"serparttypes":null,"monauthors":null,"MParts":null,"SParts":null,"hLibs":null,"langs":[{"BEntID":389282,"AbstractFlag":0,"LangID":15,"LangCode":"en","Lang":"English","DutchTerm":"Engels","LangCodeExtended":"eng"},{"BEntID":389282,"AbstractFlag":1,"LangID":15,"LangCode":"en","Lang":"English","DutchTerm":"Engels","LangCodeExtended":"eng"}],"urls":[{"URL":"https://dx.doi.org/10.1109/ICRA48891.2023.10160858","externalID":"10.1109/ICRA48891.2023.10160858","URLTypeCode":"DOI","URLID":129731,"URLTypID":13,"URLType":"DOI","URLPrefix":"http://dx.doi.org/"}],"thesterms":null,"taxterms":null,"geoterms":null,"othterms":null,"asfacodes":null,"asfa2codes":null,"thestermsFRIS":null,"taxtermsFRIS":null,"geotermsFRIS":null,"othtermsFRIS":null,"resmessage":"","complete":1,"sessions":{"newSesName":"Bouchti, Zohra, Z.","newSesDate":{"date":"2024-03-26 11:58:05.477000","timezone_type":3,"timezone":"Europe/Brussels"},"updSesName":"Bouchti, Zohra, Z.","updSesDate":{"date":"2024-04-06 09:42:12.830000","timezone_type":3,"timezone":"Europe/Brussels"}}}